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Overview 


Young people have been hit especially hard by changes in the labor market over the past decades. 
Unemployment among 1 6- to 24-year-olds increased the most of any age group during the recent 
recession, and remains more than double that among older adults. The unemployment rate is 
especially high for young people without high school diplomas. YouthBuild is one program that 
attempts to help this group, serving over 10,000 of them each year at over 250 organizations 
nationwide. Each organization provides construction-related or other vocational training, educational 
services, counseling, and leadership-development opportunities to low-income young people ages 16 
to 24 who did not complete high school. 

YouthBuild is being evaluated using a randomized controlled trial, in which eligible young people at 
participating programs were assigned either to a program group, invited to enroll in YouthBuild, or 
to a control group, referred to other services in the community. The evaluation includes 75 programs 
across the country funded by the U.S. Department of Labor or the Corporation for National and 
Community Service and nearly 4,000 young people who enrolled in the study between 2011 and 
2013. This report, the second in the evaluation, presents the program’s effects on young people 
through two and a half years. 

Main Findings 

About 75 percent of the young people assigned to the program group participated in YouthBuild, 
and about half of these participants reported that they graduated from the program within 12 months. 
YouthBuild led to a number of positive effects on young people, most consistently in the area of 
education and training. 

• YouthBuild increased participation in education and training, even though a high percentage of 
the young people in the control group also sought out and participated in education and training. 

• Overall, participants rated their experiences in YouthBuild favorably, although some program 
components were rated more highly than others. 

• YouthBuild increased the rate at which participants earned high school equivalency credentials, 
enrolled in college, and participated in vocational training. 

• Y outhBuild led to a small increase in wages and earnings at 30 months. 

• YouthBuild increased civic engagement, particularly volunteering, but had few effects on other 
measures of youth development or attitudes. 

• YouthBuild had few effects on involvement in the criminal justice system. 

The program’s interim effects on education and training are encouraging. A later report, measuring 
effects through four years, will examine whether these interim effects lead to longer-term gains in 
work and earnings. 
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Executive Summary 


Making the successful transition to adulthood has become more and more difficult for many 
American young people. Unemployment rates among young people are more than double those 
among older adults, and young people were hit especially hard by the recent recession of 2007- 
2009. 1 Finding well-paying work has become particularly challenging for young people without 
high school diplomas or equivalents. Their unemployment rate peaked at over 35 percent during 
the recession and remains high today, at over 20 percent in the second quarter of 2016. 2 Finding 
ways to reengage these young people in education and work is one of our nation’s central social 
policy challenges. 

YouthBuild is one program that attempts to help this group. YouthBuild is a federally 
and privately funded program operated at over 250 organizations nationwide, serving over 
10,000 young people each year. Each organization provides construction-related training and 
may also provide training in other in-demand industries, along with educational services, 
counseling, and leadership-development opportunities, to low-income, out-of-school young 
people ages 16 to 24. 

In 2010, the Department of Labor (DOL), with initial support from the Corporation for 
National and Community Service (CNCS), awarded a contract to MDRC and its partners Social 
Policy Research Associates and Mathematica Policy Research to conduct an impact evaluation 
of YouthBuild. The evaluation includes 75 programs across the country receiving funding from 
either DOL or CNCS, and nearly 4,000 young people who enrolled in the study between 2011 
and 2013. The evaluation will examine YouthBuild’s effects on the young people it serves for 
up to four years after they enter the study, assessing effects on a wide range of outcomes 
including education and training, work and earnings, youth development, and involvement in 
the criminal justice system. 

This report presents the interim effects of the program two and a half years after 
young people applied to YouthBuild, examining participation in the program, effects on 
education and training, and early effects on work and earnings. The findings show that 
YouthBuild led to a number of positive effects on this group of young people, including a 


*U.S. Department of Labor, Bureau of Labor Statistics, “BLS Data Viewer: (Seas) Unemployment Rate 

— 16-24 Yrs.,” 2016, website: http://beta.bls.gov/dataViewer/view/timeseries/LNS14024887; U.S. Depart- 
ment of Labor, Bureau of Labor Statistics, “BLS Data Viewer: (Seas) Unemployment Rate — 25 Yrs. and 
Over,” 2016, website: http://beta.bls.gov/dataViewer/view/timeseries/LNS14000048. 

2 U.S. Department of Labor, Bureau of Labor Statistics, “BLS Data Viewer: (Unadj) Unemployment Rate 

— Not Enrolled in School, Less Than a High School Diploma 16-24 Yrs.,” 2016, website: http://beta.bls.gov/ 
dataViewer/view/timeseries/LNU04023019Q. 
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notable increase in the rate at which they earned General Educational Development (GED) 
credentials and a small, positive effect on college enrollment. 3 YouthBuild also led to a small, 
positive effect on wages and earnings. A later report will examine the program’s effects on 
education and work after four years. 

The YouthBuild Model 

All YouthBuild programs in operation are modeled to some degree on the original YouthBuild 
program: the East Harlem Youth Action Program founded in the late 1970s in New York City. 
That original program was designed to address the complex needs of participants and their 
community with a culture of respect for young people that is still emphasized today. 

The YouthBuild model includes a mix of education, vocational training (usually train- 
ing in construction), counseling, leadership development, and community service. Eligibility is 
typically limited to out-of-school young people ages 16 to 24 who have dropped out before 
completing high school and who meet one of the following criteria: They are from low-income 
or migrant families, are in foster care or are aging out of it, are ex-offenders, have disabilities, or 
are children of incarcerated parents. 

Programs recruit or rely on word of mouth to identify interested applicants, who then go 
through assessments before enrolling such as tests of basic skills and one-on-one interviews. 
Most frequently, programs then implement a rigorous Mental Toughness Orientation, which can 
last from a single day to several weeks. Mental Toughness Orientation is designed to facilitate 
group bonding and ready recruits for the program’s activities. It also serves as a period when 
many young people are screened out because they stop attending or otherwise fail to follow 
established rules. 

Most young people who make it through Mental Toughness Orientation enroll in 
YouthBuild, are offered the program’s services, and participate for 6 to 12 months. New 
participants typically begin the program in a group with other enrollees, and that group 
alternates weekly or every few weeks between a focus on education and a focus on vocational 
training. The components of the model are intended to be integrated and designed to be 
offered together. 


3 The term “GED” is used throughout this report to indicate a high school equivalency credential, even 
though many states no longer use the official GED test to grant those credentials. Given the timing of the 
YouthBuild evaluation, most study participants would still have taken the GED exam to earn their high school 
equivalency credentials. 
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YouthBuild’s services consist of: 


• Educational services such as instruction in basic skills, remedial education, 
and alternative education leading to a high school diploma or GED. A grow- 
ing number of programs also offer services to prepare young people for post- 
secondary education. 

• Vocational training, typically construction training in which participants re- 
habilitate or build housing for low-income people. Begimiing in 2012, certain 
DOL-funded programs were authorized to provide a “construction plus” 
model, in which funding could be used to offer training for in-demand occu- 
pations outside construction. Before that date, some programs were already 
providing training for other vocations such as Certified Nursing Assistant, 
commercial driver, or infonnation technology professional. 

• Youth-development services, including leadership training and community 
service. These services are defining features of YouthBuild that are ad- 
dressed in multiple ways and serve multiple purposes. Leadership training is 
approached through structured curricula or formal and informal roles for par- 
ticipants within the YouthBuild program on committees, in the classroom, on 
work sites, or in community activities and meetings. Young people partici- 
pate in community service by constructing affordable housing and through 
other activities; this community service attends to the community’s needs 
and gives young people opportunities to practice leadership and other skills. 

• Supportive services and transition services include counseling, case man- 
agement, life-skills training, workforce preparation, follow-up services for 
one year, stipends for participation, and other fonns of support, such as help 
with transportation, child care, or housing. All of these services are designed 
to help young people address challenges that may prevent them from achiev- 
ing success in the program or beyond. 

The Evaluation 

YouthBuild is being evaluated using a random assigmnent design, in which eligible young 
people at participating programs around the country were assigned either to a program group, 
invited to enroll in YouthBuild, or to a control group, provided infonnation on other services in 
the community. Programs selected for the evaluation include a mix of those receiving funding 
from DOL and from CNCS in 2011. From the 74 programs that received grants from DOL in 
2011, 60 were randomly selected to participate in the study, and 58 were ultimately able to 
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participate. From the 24 programs that received CNCS grants above a certain amount but not 
DOL grants in 201 1, 17 were selected as suitable to participate in the study. 

These 75 programs enrolled a total of 3,929 young people into the study between Au- 
gust 2011 and January 2013. For each enrollment cycle, every program used its typical selection 
process to create a pool of applicants deemed eligible and appropriate for YoutliBuild. These 
applicants were then assigned at random to fill the available program slots or to a control group. 

The evaluation consists of three components. First, a process study examined the opera- 
tions of the YoutliBuild programs in the evaluation and the perceptions and experiences of the 
participating young people. The findings from the process study, presented in an earlier report, 
indicate that there was variation in how programs implemented the components of the model, in 
response to their local contexts. Overall, however, the participating programs implemented the 
YoutliBuild model well and faithfully. Fidelity to the YoutliBuild model was most consistent 
and highest in vocational services and varied more among programs in leadership development 
and preparation for postsecondary education. 

Second, an impact study is tracking the program and the control groups for four years 
using survey data and administrative records. The impact analysis will examine the program’s 
effects on a wide range of outcomes, including enrollment in education and educational attain- 
ment, work and earnings, involvement in the criminal justice system, family structure, and 
social and emotional development. This report presents interim findings, and longer-term 
findings will be presented in the final report. Finally, a cost-effectiveness study will estimate the 
costs of operating and running YoutliBuild and compare these costs with any positive gains that 
are achieved. The results from this analysis will be included in the final report. 

The analysis presented in this report is based on several data sources. First, surveys 
were administered to a random subset of study participants 12 and 30 months after they entered 
the study. These surveys collected information on education and training, work and earnings, 
use of services, family formation, involvement in the criminal justice system, youth develop- 
ment, and health and well-being. Second, administrative records were obtained for the full study 
sample on employment and earnings (from the National Directory of New Hires) and postsec- 
ondary enrollment (from the National Student Clearinghouse). Third, a survey of YoutliBuild 
programs provided information on program characteristics. Finally, program data on enrollment 
reported to the DOL management infonnation system are used to corroborate YoutliBuild 
participation as reported by young people on the surveys. 4 


4 A management infonnation system is a database that holds information on program operations and that 
can produce reports on every level of a program’s management. 
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The young people enrolled in the study generally fit the profile of typical YouthBuild 
participants. The majority of study participants are male (64 percent) and most are either black 
(63 percent) or Latino (15 percent). On average, study participants were nearly 20 years old 
when they entered the study, with about 70 percent older than age 18. Over 90 percent did not 
have high school diplomas or GEDs when they entered the study, and about 60 percent of them 
had left school after completing the tenth or eleventh grade. 

Findings 

• About 75 percent of the young people assigned to the program group 
participated in YouthBuild, and about half of these participants report- 
ed that they graduated from the program within 12 months. 

At 12 months, 74 percent of the program group reported ever receiving YouthBuild 
services or participating in YouthBuild activities. Those who reported participating said that 
they remained in YouthBuild for an average of 8 months, and about 50 percent of them reported 
completing the program, or graduating. About a quarter of the participants (22 percent) reported 
still being involved in YouthBuild at 12 months, and 28 percent had left the program without 
completing it. 

• Overall, participants rated their experiences in YouthBuild favorably, 
although some program components were rated more highly than 
others. 

Eighty-seven percent of participants rated their overall YouthBuild experiences favor- 
ably, meaning “very good” or “good.” They were also asked about each of the program ser- 
vices: The most highly rated services were construction or job training, counseling, and leader- 
ship training. Participants gave the YouthBuild staff favorable ratings related to understanding 
their needs and helping them to leam. They reported being slightly less satisfied with the 
services they received after leaving the program, such as assistance finding a job. 

• YouthBuild increased participation in education and training, even 
though a high percentage of young people in the control group sought 
out and participated in education and training activities. 

The program’s rigorous screening processes are designed to ensure that the only young 
people who enter the program are those who have a good chance of completing it. As a result, 
young people in both the program and control groups who made it through the screening 
process and into the study are probably more motivated and persistent than the typical young 
person who has not completed high school. By 30 months after enrollment, for example, 70 
percent of the control group had participated in education-related services. 
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Despite this high rate of control group participation, however, young people in the 
program group were more likely than their control group counterparts to have participated in 
education and training, especially GED preparation, vocational training, and a variety of 
youth-development activities. For example, 75 percent of the program group reported partici- 
pating in an education-related activity during the first 12 months, compared with 57 percent 
of the control group. 

• YouthBuild increased GED receipt and enrollment in two-year colleges. 

Most young people who entered the program had not completed high school, and a cen- 
tral goal of the program is to help these young people earn GEDs. The program did have a 
sizable effect on GED receipt. By 30 months, about 18 percent of the young people in the 
control group reported earning GEDs, compared with 3 1 percent of the program group, an 
increase of 14 percentage points (see Table ES.l). This estimated effect includes all young 
people in the program group, not accounting for the fact that 25 percent of them never partici- 
pated in YouthBuild. The effect on young people who did participate in YouthBuild is 19 
percentage points. 

Getting a GED by itself may increase college attendance, but many YouthBuild pro- 
grams explicitly focus on helping young people make the transition to postsecondary education. 
By 30 months, 22 percent of the young people in the program group reported enrolling in a two- 
year community college at some point since they entered the study, compared with 17 percent 
of the control group, an impact of 5 percentage points. This increase in survey-reported college 
attendance is corroborated by administrative records from the National Student Clearinghouse. 

• YouthBuild increased participation in vocational training and led to a 
small increase in the receipt of training certificates. 

Vocational training, primarily in construction, is another central part of the YouthBuild 
model. YouthBuild seems to have increased participation in vocational training both during the 
program period and afterward. In the first year, the program group was more likely than the 
control group to have participated in a job-skills training program. Many members of the 
program group did so at a YouthBuild location. When survey respondents were asked at 30 
months about formal enrollment in vocational training at a technical, business, or trade school, 
YouthBuild still had an impact: 31 percent of the program group reported having enrolled in 
such training since entering the study, compared with 20 percent of the control group. Very few 
young people reported obtaining vocational certificates by Month 30: 4 percent of the program 
group and 2 percent of the control group. 
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Table ES.l 


Impacts on Key Outcomes at 30 Months 


Outcome 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

Education and training (%) 

Earned a GED 

31.2 

17.5 

7 H=H=H= 

Enrolled in vocational school since random assignment 

30.8 

20.4 

10.3 *** 

Received a trade license/training certificate 

4.1 

2.1 

2.0** 

Enrolled in postsecondary courses since random assignment 

23.6 

18.1 

5.6*** 

Work and earnings 

Employed at Month 30 (%) 

44.4 

44.8 

-0.4 

Average weekly earnings ($) 

150.2 

134.5 

15.7* 

Youth development 

Civic engagement 3 (%) 

92.2 

88.6 

3.6*** 

Self-esteem scale b 

3.3 

3.3 

0.0 

Criminal justice involvement (%) 

Arrested since random assignment 

27.6 

26.4 

1.3 

Convicted since random assignment 

15.7 

14.1 

1.6 

Sample size (total = 2,808) 

1,830 

978 



SOURCE: MDRC calculations based on responses to the 30-month survey. 

NOTES: Statistical signficance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

‘‘"Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, 
and being involved in politics or local community activities. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range 
from 1 = "strongly disagree" to 4 = "strongly agree," where higher scores indicate higher levels of self- 
esteem. Responses to the 10 items are averaged. 


• YouthBuild led to a small increase in employment rates during Year 2, 
and a small increase in survey-reported wages and earnings at 30 
months. 

The opportunities for education and training in YouthBuild and the impact the program had on 
GED receipt should help participating young people to find jobs after completing the program. 
At the time of the 30-month survey, just over 40 percent of respondents in the program group 
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and the control group were employed; the program had no effect on employment reported on 
the survey. However, young people in the program group reported earning higher wages. The 
program led to an increase of 3 percentage points in the proportion of young people who 
reported earning at least $10 per hour at their current jobs (not shown in the table). These higher 
hourly wages also translated into higher weekly earnings. 

Data from the unemployment insurance system (not shown in the table) present a fuller 
picture over time. During the first year, while young people were participating in YouthBuild, 
the program led to a reduction in employment and earnings relative to the control group (whose 
members were less likely to be enrolled in a program and thus were more available to work). 
However, by Year 2, there was no difference in earnings between the two groups, and the 
program group had somewhat higher employment rates. Longer-term follow-up presented in the 
final report will assess whether the program group pulls farther ahead over time. 

• YouthBuild increased civic engagement, particularly volunteering, but 
had few effects on other measures of youth development or attitudes. 

YouthBuild has been a leader in integrating youth development into its programs by 
promoting leadership and community service. In addition, the programs receiving CNCS 
funding strongly emphasize community service. At 30 months, members of the YouthBuild 
group were more likely than members of the control group to report that they had volunteered 
or been involved in politics or local community activities. The largest impact occurred on 
volunteering: 54 percent of the program group reported volunteering, compared with only 31 
percent of the control group (not shown). 

A number of questions were included in the surveys to capture other aspects of youth 
development, such as self-esteem, self-confidence, feelings about the future, and feelings of 
social support. YouthBuild had no effect on responses to these questions. Other research on 
youth programs suggests that it is difficult to create lasting changes in many of these measures. 5 

• YouthBuild had few effects on involvement in the criminal justice 
system. 

Participation in YouthBuild may lead to a number of positive changes for participants 
through effects on education, work, and youth development. For example, program participa- 
tion should reduce the amount of time that young people spend hanging out idle, and thus 
reduce their opportunities to engage in risky and unhealthy activities. Yet at 30 months, the 
program had no effect on arrest or conviction rates. About 25 percent of the young people in the 


5 Jacquelynne Eccles and Jennifer Appleton Gootman, eds., Community Programs to Promote Youth 
Development (Washington, DC: National Academies Press, 2002). 
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study (in the program and control groups) reported that they had been arrested since they 
entered the study. Moreover, the program did lead to increases of about 5 percentage points in 
rates of self-reported alcohol and drug use (not shown in the table). It will be important to track 
whether these effects persist. 

Conclusion 

The findings presented here show that YouthBuild led to a number of positive effects, most 
consistently in the area of education and training. The program led to notable increases in GED 
receipt and participation in vocational training, and positive but small effects on college enroll- 
ment. A later report will examine whether these early increases in education and training lead to 
longer-term gains in work and earnings. 

The research to date on youth programs has provided mixed results. Among the numer- 
ous programs studied, most increased the rate at which participants earned GEDs or other 
credentials, and had mixed effects on work and earnings. 6 The effects found here follow suit. 
They are similar to or more positive than findings from other nonresidential programs, but 
somewhat smaller than the interim effects found for more intensive, residential programs, such 
as Job Corps and National Guard Youth ChalleNGe. 

Although it is too early to make judgements about YouthBuild’s effects overall, the 
findings here suggest that there may be room for improvement in at least a few areas: helping 
participants find jobs, preparing them for the transition to postsecondary education, and main- 
taining contact with them after they leave the program. In some cases, many programs are 
already beginning to make changes that are likely to strengthen their impact — for example, 
many are beginning to emphasize more heavily their efforts to create connections with colleges. 
Similarly, many programs are beginning to offer vocational training not only in construction but 
in other areas as well, for example in infonnation technology and health care. 


6 See George Cave, Hans Bos, Fred Doolittle, and Cyril Toussaint, JOBSTART: Final Report on a Pro- 
gram for School Dropouts (New York: MDRC, 1993); Jo Ann Jastrzab, Julie Masker, John Blomquist, and 
Larry Orr, Evaluation of National and Community Service Programs. Impacts of Service: Final Report on the 
Evaluation of American Conservation and Youth Service Corps (Washington, DC: Corporation for National 
Service, 1996); Cynthia Miller, Johannes M. Bos, Kristin E. Porter, Fannie M. Tseng, and Yasuyo Abe, The 
Challenge of Repeating Success in a Changing World: Final Report on the Center for Employment Training 
Replication Sites (New York: MDRC, 2005); Cristofer Price, Julie Williams, Laura Simpson, JoAnn Jastrzab, 
and Carrie Markovitz, National Evaluation of Youth Corps: Findings at Follow-Up (Washington, DC: 
Corporation for National and Community Service, Office of Strategy and Special Initiatives, 2011); Peter Z. 
Schochet, John Burghardt, and Sheena McConnell, “Does Job Corps Work? Impact Findings from the 
National Job Corps Study,” American Economic Review 98, 5 (2008): 1,864-1,886; Megan Millenky, Dan 
Bloom, Sara Muller-Ravett, and Joseph Broadus, Staying on Course: Three-Year Results of the National 
Guard Youth ChalleNGe Evaluation (New York: MDRC, 2011). 
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Finally, when interpreting the program’s effects, it is important to keep in mind two 
facts about the context. First, the effects presented here are estimated by comparing YouthBuild 
participants with similarly motivated young people who sought out other services in their 
communities. They are not estimated by comparing YouthBuild participants with a group who 
received no services. For this reason, the effects presented here are not of YouthBuild compared 
with no services, but of YouthBuild compared with other services for young people. Second, 
the follow-up period for this report was a time when many areas were still recovering slowly 
from the national recession that began in 2007, and when the national unemployment rate was 
still high for less-educated young people. One of YouthBuild’s goals is to help these young 
people get an early advantage in the labor market. A later report will assess whether the pro- 
gram achieves that goal. 
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Chapter 1 

Introduction 


Changes in the labor market over the past several decades have made it tougher for many 
American workers to find and maintain well-paying jobs. Wages for those with less education 
have fallen over time, for example, contributing to an increase in inequality that has been well 
documented. 1 The recent recession worsened these trends. The unemployment rate was 15 
percent at the end of 2009 for those without a high school diploma, compared with less than 5 
percent for those with a college degree. 2 

Young people have been hit especially hard by these changes. Unemployment for indi- 
viduals ages 16 to 24 increased the most during the recent recession, peaking at just over 19 
percent in late 2009. 3 Although the rate has fallen since then, the youth unemployment rate 
remains more than double that for adults. 4 The unemployment rate for young people without 
high school diplomas is especially high, at over 20 percent in the second quarter of 20 16. 5 Rates 
of unemployment are even higher for black and Latino young men. 6 

As a result, it has become more and more difficult for many young people to make the 
transition to adulthood successfully. Those without high school diplomas — and there are 
nearly three million of them today — face particular challenges. Nearly 40 percent of them 
never earn high school credentials and even fewer pursue further education, even though 
postsecondary training is increasingly viewed as a necessary step on the path to a good job. 7 
Finding ways to reengage these young people in education and work and help them become 
thriving adults is one of our nation’s central social policy challenges. 

YouthBuild is one program that attempts to help this group. YouthBuild is a federally 
and privately funded program operated by over 250 organizations nationwide, serving over 
10,000 young people each year. Each organization provides construction-related training and 
may also provide training in other in-demand industries, along with educational services, 
counseling, and leadership-development opportunities, to low-income, out-of-school young 
people ages 16 to 24. The vast majority of programs provide construction training as their 


'Piketty and Saez (2003), with updated tables at http://eml.berkeley.edu/~saez. 
2 U.S. Department of Labor, Bureau of Labor Statistics (2015). 

3 U.S. Department of Labor, Bureau of Labor Statistics (2016b). 

4 U.S. Department of Labor, Bureau of Labor Statistics (2016b, 2016c). 

5 U.S. Department of Labor, Bureau of Labor Statistics (2016a). 

6 U.S. Department of Labor, Bureau of Labor Statistics (20 1 6d). 

7 Hurst, Kelly, and Princiotta (2004). 
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vocational instruction; in these programs participants work on renovating or constructing 
housing for low-income or homeless people. YoutliBuild distinguishes itself from other pro- 
grams serving young people without high school diplomas through a program environment that 
emphasizes youth development and leadership, capitalizing on participants’ strengths, and 
empowering participants to take responsibility for their lives. 

In 2010, the Department of Labor (DOL), with initial support from the Corporation for 
National and Community Service (CNCS), awarded a contract to MDRC and its partners, 
Social Policy Research Associates and Mathematica Policy Research, to conduct an impact 
evaluation of YoutliBuild programs funded by DOL or CNCS in 2011. The evaluation includes 
75 programs across the country and nearly 4,000 young people who enrolled in the study 
between 2011 and 2013. 

This report is the second in the evaluation. 8 It presents the program’s interim effects on 
young people two and a half years after they entered the study. The report examines effects on a 
range of outcomes, including participation in education and training, educational attainment, 
work and earnings, involvement in the criminal justice system, and civic engagement. 

In sum, the findings presented here show that YoutliBuild led to a number of positive 
effects, most consistently in the area of education and training. The program led to notable 
increases in the rate at which young people earned General Educational Development (GED) 
credentials and in their participation in vocational training, and positive but small effects on 
postsecondary enrollment. 9 A later report will examine whether these early increases in educa- 
tion and training lead to longer-term gains in work and earnings. By Month 30, the program had 
a small positive effect on earnings and mixed effects on employment rates. The program also 
increased civic engagement, particularly volunteering, but had no effects on other measures of 
youth development. 


YouthBuild 

YoutliBuild started in the late 1970s with one program in East Harlem, called the Youth Action 
Program (YAP). YAP allowed young people to improve their community by renovating and 
building housing while at the same time giving them the education and job training they needed. 
In the late 1980s and early 1990s, other programs modeled on YAP were developed under the 


8 The first report documented implementation at the 75 programs in the study (Wiegand et al., 2015). 

9 The term “GED” is used throughout this report to indicate a high school equivalency credential, even 
though many states no longer use the official GED test to grant those credentials. Given the timing of the 
YouthBuild evaluation, most study participants would still have taken the official GED exam to earn their high 
school equivalency credentials. 
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name “YouthBuild.” To support these replication efforts, staff members from YAP founded 
YouthBuild USA in 1990 to provide technical assistance and training to new YouthBuild 
programs. In 1992, under the umbrella of YouthBuild USA, a number of local YouthBuild 
programs came together to form the YouthBuild Affiliated Network made up of programs that 
agreed to uphold certain standards for performance and program design and to support advoca- 
cy efforts on behalf of the program and low-income young people. 

The expansion of the program was initially supported by private grants and then later by 
the federal government. In the early 1 990s, federal legislation allocated funds to be granted to 
YouthBuild programs through an annual, competitive process under the auspices of the U.S. 
Department of Housing and Urban Development. In 2007, responsibility for YouthBuild was 
transferred to DOL’s Employment and Training Administration. Each year, DOL awards grants 
based on a competitive review process that assesses past performance and community needs. 
The grants typically range in size from $700,000 to $1,100,000 and are intended to cover two 
years of program services for one or more cohorts of young people and 9 to 12 months of 
follow-up services. 10 Grantees are also required to raise nonfederal funds to match 25 percent of 
the DOL grants they receive. 

The YouthBuild network also receives funding from a variety of other public and pri- 
vate sources through grants to YouthBuild USA. For example, about 70 YouthBuild programs 
nationwide receive annual funding from CNCS through its grant to YouthBuild USA. These 
programs, called YouthBuild AmeriCorps programs, strongly emphasize community service 
and postsecondary enrollment. 1 1 Other sources of funding include state appropriations, educa- 
tion funding (based on average daily attendance), and foundation grants, among others. 

The programs across the country are quite diverse in structure and size. Some programs 
are community-based organizations or faith-based organizations, while others are run by local 
government agencies or educational institutions. In addition, at least 40 YouthBuild programs 
are now diploma-granting schools. While the average program serves between 30 and 40 young 
people per year, some are quite small, serving 8 to 10 young people, while others serve 75 or 
more per year. 12 


10 A “cohort” is a group of participants who join a program at the same time and move through it together. 
U A distinguishing feature of YouthBuild AmeriCorps programs is that participants are eligible for educa- 
tion awards when they complete YouthBuild. These awards range from about $1,175 to $5,500 depending on 
participants’ hours of service and other activities. 

12 One program in the evaluation served more than 200 young people each year. 
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The Model 

The YouthBuild model includes a mix of education, vocational training (typically in 
construction), counseling, leadership development, and community service. Eligibility is usually 
limited to out-of-school young people ages 16 to 24 who have dropped out before completing 
high school and who meet one of the following criteria: They are from low-income or migrant 
families, are in foster care or are aging out of it, are ex-offenders, have disabilities, or are 
children of incarcerated parents. 13 

Programs recruit or rely on word of mouth to identify interested applicants, who then go 
through assessments before enrollment such as tests of basic skills and one-on-one interviews. 
Most programs then implement a rigorous Mental Toughness Orientation (MTO), which can 
last from a single day to several weeks. MTO is designed to facilitate group bonding and ready 
recruits for the program’s activities. According to YouthBuild staff members, the top four 
activities conducted during MTO were team-building exercises, life-skills training, leadership 
development and community service, and academic work, all of which are also activities that 
young people participated in after officially enrolling in YouthBuild. 14 It also serves as a period 
when young people are screened out because they stop attending or otherwise fail to follow 
established rules. 

Most young people who make it through MTO enroll in YouthBuild, are offered the 
program’s services, and participate for 6 to 12 months. New participants typically begin the 
program with a cohort of other new young people, and the cohort alternates between education 
and vocational training. For example, many programs use a weekly rotation in which young 
people participate in education one week and vocational training the next. The components of 
the model are intended to be integrated and are designed to be offered together, which distin- 
guishes YouthBuild from other youth programs that may offer some of the same services. 

YouthBuild’s services consist of: 

• Educational services such as instruction in basic skills, remedial education, 
and alternative education leading to a high school diploma or GED. A grow- 
ing number of programs also offer services to prepare young people for post- 
secondary education. 15 


1 3 A low-income family is defined as one whose income falls below 80 percent of median family income in 
the local area. 

14 See Wiegand et al. (2015) for more details on the content and structure of MTO. 

15 Efforts to promote college enrollment have been supported by several grants to YouthBuild USA and are 
also authorized and encouraged by DOL. 
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• Vocational training, typically training in construction in which participants 
rehabilitate or build housing for low-income people. Beginning in 2012, cer- 
tain DOL-funded programs were authorized to provide a “construction plus” 
model, in which funding could be used to offer training for in-demand occu- 
pations outside construction. Before that date, some programs were already 
providing training for other vocations such as Certified Nursing Assistant, 
commercial driver, or information technology professional. 

• Youth-development services, including leadership training and community 
service. These services are defining features of YouthBuild that are ad- 
dressed in multiple ways and serve multiple purposes. Leadership training is 
approached through structured curricula or formal and informal roles for par- 
ticipants within the YouthBuild program on committees, in the classroom, on 
work sites, or in community activities and meetings. Young people partici- 
pate in community service by constructing affordable housing and through 
other activities; this community service attends to the community’s needs, 
teaches the value of helping others, and provides opportunities for young 
people to practice leadership and other skills. 

• Supportive services and transition services include counseling, case man- 
agement, life-skills training, workforce preparation, follow-up services for 
one year, stipends for participation, and other forms of support, such as help 
with transportation, child care, or housing. All of these services are designed 
to help young people address challenges that may prevent them from achiev- 
ing success in the program or beyond. 

Its focus on youth development distinguishes YouthBuild from more traditional em- 
ployment programs for young people. YouthBuild reflects a movement to empower young 
people, advocate for them, foster their civic engagement and activism, and encourage them to 
take on roles of responsibility and leadership in their personal lives and broader communities. 

Together, the combination of services is hypothesized to create a number of positive 
changes for participants, which are shown in in the rightmost boxes in Figure 1.1. In the short 
term, YouthBuild aims to increase participants’ basic skills and help them earn a GED or high 
school diploma. Young people in the program can also accumulate work experience at the work 
sites, earn training credentials, and find jobs. Less tangibly, YouthBuild aims to stimulate 
lasting changes in attitudes that will keep young people on a positive trajectory and increase 
their civic engagement. In the longer term, YouthBuild aims to see its participants enroll in and 
complete college, maintain stable employment, earn more money, and have less involvement 
than their peers in the criminal justice system. 
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Figure 1.1 

Participant-Level YouthBuild Conceptual Framework 
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The bottom of Figure 1 . 1 shows that a variety of contextual factors can influence a par- 
ticipant’s experience in YouthBuild and subsequent outcomes. A number of program features 
could also affect outcomes and impacts — chiefly the program’s fidelity to the YouthBuild 
model. This report examines whether programs of varying fidelity have different effects, and 
whether YouthBuild has different effects on participants with different characteristics. For 
example, in visits to the programs conducted for the implementation study, staff members often 
mentioned that older participants seemed more ready to benefit from YouthBuild’s services 
than their younger counterparts. This report therefore compares YouthBuild’s effects on 
participants who were older when they entered the study (those 20 and older) with its effects on 
those who were younger (those under age 20). 

Apart from fidelity to the YouthBuild model, a range of other factors might also affect 
outcomes, such as the intensity of Mental Toughness Orientation, the strength of a program’s 
postsecondary preparation, or the strength and tenure of its leaders. A formal analysis of how 
such program features influence programs’ impacts will appear in the final report. 

The local context might also affect program impacts. For example, a program may 
have a smaller measurable effect in areas where there are a range of other options for educa- 
tional and vocational services, since young people in the control group would have access to 
these services. In addition, the local economy might have important effects. As documented 
in the earlier implementation report, the programs participating in the evaluation are generally 
located in higher-poverty areas facing myriad challenges. Staff members described their 
neighborhoods as having dilapidated housing, little public transportation or other infrastruc- 
ture, few job opportunities, and poor schools. 16 In interviews for the implementation study (in 
2012 and 2013), they also described the effects of the Great Recession — a halt in construc- 
tion, business closures, and layoffs — and how these conditions hit hardest the young people 
they serve. Conditions had improved by the time they were interviewed, although not by 
much. Unemployment rates have fallen somewhat since 2009, but they still remain quite high 
for young people (see Figure 1 .2). 

The local economy can affect program impacts in two important ways. First, it affects 
what the program can offer young people. For example, the downturn in the housing market 
made it more difficult for programs to find new construction projects and thus good training 
opportunities for participating young people. Second, the economy, locally and more broadly, 
affects how participants fare after the program. Young people enrolled in the study between late 
201 1 and early 2013, so the follow-up period covered in this report ranges from 2012 through 
mid-2015. Unemployment rates were still quite high during this time, particularly for young 


16 Wiegand et al. (2015). 
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Percentage unemployed 


Figure 1.2 

Unemployment Rates by Age 



^^—16 to 24 years old 25 years old or older 


SOURCE: U.S. Department of Labor, Bureau of Labor Statistics (2016a). 


people. It is not clear how the effects of an education and training program might vary with the 
economy. Recent research suggests that labor-market programs can have bigger effects in 
slower economic times, although the depth and longevity of the Great Recession was unlike any 
previous downturn over the last 75 years, and this research did not examine the question for 
young people specifically. 17 Beyond unemployment rates, however, structural changes in the 
economy over the past several decades have led to reduced earnings opportunities for many 
workers. People with less education are more and more likely to work in low-paying service- 
sector jobs, and wages have even fallen in the higher-paying jobs traditionally available to less- 


17 Card et al. (2015). 
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educated workers. 18 It will be important to keep this economic context in mind when interpret- 
ing the findings. 

There have been a number of studies of YouthBuild over the past 20 years, although 
none were conducted as randomized controlled trials. 19 Most studies have either documented 
program implementation or tracked YouthBuild graduates to assess how they fare after leaving 
the program. Ferguson and Snipes, for example, conducted a fomiative evaluation from 1991 to 
1994 of the first five YouthBuild replication programs, documenting the challenges programs 
faced in achieving high fidelity to the model, and the essential features of the model that help 
young people succeed. Another study of YouthBuild graduates selected primarily from a subset 
of established YouthBuild programs found that YouthBuild graduates fared relatively well after 
leaving the program, with a majority either enrolled in school or training, or working and 
earning above a certain wage per hour. 20 

The Evaluation 

YouthBuild is now being evaluated using a random assignment design, in which eligi- 
ble young people at participating programs around the country were assigned either to a 
program group, invited to enroll in YouthBuild, or to a control group, provided information on 
other services in the community. Young people assigned to the control group could not enroll in 
YouthBuild at the participating program for two years. The next chapter examines how many 
young people in the program group actually participated in YouthBuild services, and also the 
types of services in which the control group participated. 

The evaluation consists of three components: 

• Process study. This study examined the operations of the YouthBuild pro- 
grams in the evaluation and the perceptions and experiences of the participat- 
ing young people. The study assessed each program’s fidelity to the Youth- 
Build model. Findings from the process study were presented in an earlier 
report. 21 

• Impact study. This study will track both the program and the control groups 
for four years using survey data and administrative records. 22 The impact 
analysis will examine the program’s effects on a wide range of outcomes, in- 

18 Keamey, Hershbein, and Boddy (2015). 

19 See, for example, Ferguson and Snipes (1994); Hahn, Leavitt, Horvat, and Davis (2004); Mitchell et al. 
(2003); Hahn and Leavitt (2007); and Cohen and Piquero (2009). 

20 Hahn, Leavitt, Horvat, and Davis (2004). 

21 Wiegand et al. (2015). 

~ 2 Administrative records are data collected primarily for the management of programs and public services. 
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eluding enrollment in education and educational attainment, work and earn- 
ings, involvement in the criminal justice system, family structure, and social 
and emotional development. This report presents interim findings, and 
longer-term findings will be presented in the final report. 

• Cost-effectiveness study. This analysis will estimate the costs of operating 
and running YouthBuild and compare these costs with any positive gains that 
are achieved. The results from this analysis will be included in the final 
report. 

Program Selection 

The programs selected for the evaluation include a mix of those receiving funding from 
DOL in 2011 and those receiving funding from CNCS. The first selection pool included the 
group of 74 programs that received grants from DOL (and in some cases CNCS as well, 
although this report refers to them as “DOL-funded programs”). Three of these programs were 
deemed unsuitable for the study and were dropped from the pool. 23 From the remaining 71 
programs, 60 programs were randomly selected to participate in the study. 24 After discussions 
with program staff members and DOL, 2 of the 60 selected programs were subsequently 
dropped from the evaluation. 25 See Appendix A for more infonnation about program selection. 

The remaining programs were selected from a group of programs that did not receive 
DOL funding in 2011 but received relatively large grants from CNCS, through its National 
Direct Grant to YouthBuild USA. 26 DOL and CNCS chose to include the CNCS-funded 
programs in the evaluation in order to examine whether DOL-funded programs have different 


^Interviews with staff members at these three programs indicated that young people assigned to a control 
group would be likely to receive services that were very similar to YouthBuild services, which would provide a 
poor test of YouthBuild’s effects. These programs accounted for only 4 percent of the expected YouthBuild 
enrollment among all grantees. 

24 Budget considerations prohibited selecting all 7 1 programs for the study. Programs were selected using 
probability-proportional-to-size sampling, in which larger programs representing more young people had a 
greater probability of selection. Selecting programs in this way meant that each program slot, or young person, 
had an equal chance of selection. 

25 Several discussions with program and DOL staff members revealed that random assignment was not 
feasible at these two programs because they would not be able to enroll study participants during the evalua- 
tion’s intake period. 

26 Discussions with DOL and YouthBuild USA staff members suggested that the study should draw from 
those programs that received grants of $95,000 or more from CNCS. Of the 40 programs that received CNCS 
funding but not DOL funding in 201 1, 24 programs received grants of $95,000 or more. 
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impacts than CNCS-funded programs. 27 Of the 24 programs that received relatively large 
CNCS grants, 7 were deemed not suitable for the evaluation, leaving 17 CNCS-funded pro- 
grams in the study. 28 

In total, 75 programs were selected for the study: 58 DOL-funded programs and 17 
CNCS-funded programs. Although effects are examined across all programs combined, it is 
important to remember that the group of CNCS-funded programs is not a random sample of 
such programs. However, as mentioned later, the programs in the evaluation look very similar 
to the broader population from which they were selected. 

Program and Study Intake 

The participating programs enrolled 3,929 young people into the study between August 
2011 and January 20 13. 29 The study team worked with each program to implement random 
assignment during one or more of its enrollment cycles during this period. The general proce- 
dure was to determine the number of suitable applicants for the program and offer available 
program slots to these young people through a lottery-like process. The programs used their 
normal selection processes as much as possible to create the eligible pool of applicants. Figure 
1.3 illustrates a typical YoutliBuild selection process, though the details varied slightly from 
program to program. Random assignment would then occur among that smaller, eligible pool of 
applicants. 

As shown in the figure, the first step in intake was recruitment, which is typically a ma- 
jor activity for most programs. It involved considerable staff effort, in part because YoutliBuild 
programs usually recruited many more applicants than they needed to fill their open slots. 
Excess applicants were needed because some applicants were determined to be ineligible or 
unsuitable for the program during screening, and others dropped out during screening. See 
Chapter 4 of the earlier implementation report for more detail on recruitment and screening. 30 

Once recruited, the next step was to determine whether a young person met additional 
eligibility requirements beyond those listed above. Staff members reviewed young people’s 


~ 7 The funding-source distinction is not very clear-cut, however, since many of the DOL-funded programs 
also received CNCS grants. Similarly, several of the 17 CNCS-funded programs in the study received DOL 
grants in the subsequent year. 

28 Four of the programs planned to shut down in the coming year, and three indicated that young people in 
the control group would be likely to receive services similar to YouthBuild services. 

29 Three programs were unable to conduct random assignment during the intake period because their en- 
rollment numbers were too low. These programs were included in the process study but are not included in the 
impact study. 

30 Wiegand et al. (2015). 
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Figure 1.3 

Typical YouthBuild Selection Process 
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b Although YouthBuild is a program aimed at high school dropouts, 25 percent of participants in programs funded by DOL can be young 
people who are not high school dropouts, who have high school diplomas, or who are not in one of the target populations, as long as they are 
deficient in basic skills or are referred to a high school diploma-granting YouthBuild program by another high school. 




application fonns and conducted various assessments and interviews to determine whether 
applicants met these additional requirements. Most commonly, applicants had to live within 
certain geographical boundaries and have minimum math and reading scores on assessments of 
basic skills. At some point during the screening process, the majority of study programs 
administered an academic skills test, usually the Test of Adult Basic Education or the Compre- 
hensive Adult Student Assessment Systems. Programs were divided in how they used these 
tests: Roughly half used them to screen out applicants who did not meet minimum test-score 
requirements, and the rest used them for diagnostic purposes, to establish a baseline for each 
applicant’s skills and to determine what academic services that applicant needed. 31 Some 
programs used additional criteria, such as not having a substance abuse problem, to determine 
whether a young person was appropriate for the program. Applicants who met these criteria 
were often described as having demonstrated “readiness,” or a motivation to make positive 
changes in their lives. 

After the initial screening process, young people were invited to MTO. As a result of 
the intensive application and screening, nearly half of the young people who turned in 
applications to the study programs did not receive invitations to MTO, either because they 
dropped out during the screening process or because the program decided they were not 
suitable for YouthBuild. 32 

MTO is designed to determine young people’s willingness to change, to gauge their in- 
terest and motivation, to build teamwork while they get to know one another, and to introduce 
them to the specifics of the YouthBuild program. The duration and intensity of MTO varied 
quite a bit; on average it lasted for 10 days and for seven hours per day. According to staff 
members at the study programs, the top four activities conducted during MTO were team- 
building exercises, life-skills training, leadership development and community service, and 
academic work, all of which are also activities that young people participated in after officially 
enrolling in YouthBuild. 33 Although it is an important step in the process, DOL does not count 
young people who do not complete MTO, and thus do not go on to receive core services, as 
program participants. 

MTO also served as an additional form of screening. During MTO, staff members 
might determine that a young person was not ready for YouthBuild or not a good fit for the 
program and ask that person to stop attending, or a young person might stop attending and 


3 'Programs that required a minimum score set that minimum at a sixth-grade reading level, on average. 
See Wiegand et al. (2015) 

32 For example, a young person may not have fully participated in preprogram activities, or may have 
caused a disturbance while attending. 

33 Wiegand et al. (2015). 


13 



therefore select him- or herself out of participation. An average of one in four recruits who were 
invited to MTO did not complete it. 34 

An important issue for the study team was the timing of random assigmnent at each 
program. Would young people deemed eligible and appropriate for YouthBuild be randomly 
assigned to the program or the control group before MTO, sometime during MTO, or after they 
had completed MTO? One argument for conducting random assignment before MTO was that 
many staff members considered that orientation to be an important part of YouthBuild. Con- 
ducting random assigmnent before the orientation would ensure that young people assigned to 
the control group did not receive any of the YouthBuild program. However, an argument for 
conducting random assignment after MTO was that many young people drop out of this 
orientation before going on to receive YouthBuild’s core services. Conducting random assign- 
ment before the orientation would therefore also mean that many young people assigned to the 
program group would never receive YouthBuild services, which would hinder the study’s 
ability to detect program impacts. Ultimately, the decision about when to conduct random 
assigmnent was made by each program, with input from the study team. Most programs (81 
percent of programs, representing 75 percent of study participants) opted to conduct random 
assignment before MTO or during its first few days. 35 

In order to ensure an adequate number of young people for available slots, the study 
team aimed for a random assigmnent ratio in which 60 percent of eligible young people would 
be assigned to the program group and 40 percent to the control group. In practice, some pro- 
grams had difficulty securing enough excess applicants to meet this 60:40 ratio and were 
allowed to deviate from it if necessary. On average, 69 percent of eligible young people were 
randomly assigned to the YouthBuild group and 31 percent were assigned to the control group. 
Only about 12 percent of programs (representing 16 percent of study participants) used a ratio 
above 80:20. 

Data and Methods 

The analysis presented in this report is based on several data sources. First, surveys 
were administered to a random subset of study participants 12 and 30 months after they entered 
the study. The surveys obtained information on participation in education and training, educa- 
tional attainment, work, family formation, and involvement in the criminal justice system. Both 
surveys achieved 80 percent response rates with very small differences in response rates 

34 Wiegand et al. (2015). 

35 The timing of random assignment was found to have no effect on the percentage of young people in the 
program group who ultimately went on to receive the core YouthBuild services. Similarly, program impacts for 
those programs that conducted random assignment before MTO or within its first few days were similar to the 
impacts for programs that conducted random assignment later (not shown). 
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between the program and control groups. 36 A total of 2,845 participants provided responses to 
the 12-month survey, and 2,808 participants provided responses to the 30-month survey. 

Administrative records on employment and earnings were obtained for the full sample 
If om the National Directory of New Hires, which provides quarterly wage data for existing 
workers in employment covered by the unemployment insurance system. These data will miss 
employment that is not covered by unemployment insurance, including informal work and self- 
employment. Data on postsecondary enrollment were obtained for the full sample from the 
National Student Clearinghouse, which tracks enrollment and degree receipt nationally. Al- 
though it captures over 90 percent of postsecondary enrollment in the United States, its covera- 
ge varies for different types of institutions. Its coverage is highest for public institutions and 
lowest for for-profit institutions. 37 Thus, both types of records are important complements to the 
surveys, but they may miss some types of employment and postsecondary enrollment. 

A grantee survey was administered in fall 2012 to all 110 YouthBuild programs funded 
by DOL, CNCS, or both during the evaluation period, including those programs selected for the 
study. The survey asked administrators to provide information about programs’ years in 
operation, funders, operating budgets, staff structures and staff experience levels, construction- 
work-site characteristics, recent recruitment and enrollment experiences, stipends, and program- 
component characteristics. These data are used to compare the study programs with the larger 
population of programs from which they were drawn. 

Finally, all programs receiving DOL funding are required to enter data on participant 
characteristics, service receipt, and outcomes into the DOL YouthBuild management infor- 
mation system. 38 The study used these data to assess the reliability of the findings on Youth- 
Build participation shown in the surveys. All programs receiving CNCS funding entered 
quarterly data into a separate management information system. While these data do not provide 
detailed infonnation on participation, they do indicate fonnal enrollment status. 

Because young people were randomly assigned either to the program group or to the 
control group, the effects of the program can be estimated as the differences between the two 
groups’ outcomes. (Appendix Table A.2 presents a comparison of the characteristics of the 
program and control groups, showing that the two groups were similar on average when they 
enrolled in the study.) These differences between the full program and control groups (often 
referred to as “intent-to-treat” effects) are the main focus of this report and represent the effect 
of the program on the average outcomes of young people assigned to the program group, 

36 Appendix B includes a detailed analysis of survey response rates. 

37 Dynarski, Hemelt, and Hyman (2015). 

38 A management information system is a database that holds information on program operations and that 
can produce reports on every level of a program’s management. 
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whether or not they participated in YouthBuild. Occasionally the text will mention “impacts per 
participant” (sometimes referred to in other research as “treatment-on-the-treated” effects), 
which represent the effects of the program on those young people in the program group who 
actually participated in YouthBuild. Effects per participant are estimated by dividing the effects 
on the full program group by the fraction of the program group who participated in YouthBuild. 

Impacts are estimated for each outcome using regression models in which the outcome 
of interest is regressed on an indicator for program status and several variables measured at or 
before the time of random assignment. These additional variables improve the precision of the 
impact estimates; they include the participant’s age, sex, education level, parent status, and 
race/ethnicity. Variables are also included for each individual program in order to account for 
variation in the random assignment ratio among participating programs. See Appendix A for 
more detail. 

Finally, as the number of outcomes that are examined increases, the probability of ob- 
taining impacts that are statistically significant simply by chance also increases. Although the 
estimates in this report are not formally adjusted to account for multiple hypothesis testing, the 
analysis does attempt to address this risk by limiting the number of outcomes examined. In 
addition, effects that do not appear to be part of a larger pattern are given less emphasis in the 
discussion. 


Characteristics of Participating Programs and Young People 

Study Programs 

Table 1.1 presents selected characteristics of the 75 programs in the study. (See the pro- 
cess report for more detail.) Most programs were operated by nonprofit organizations, most of 
which were local or regional nonprofits, including community development corporations, 
community action agencies, and local American Job Centers. 39 

The study programs were spread across 29 states, the U.S. Virgin Islands, and Washing- 
ton, DC, and along a continuum from densely populated urban centers to rural areas (see Figure 
1.4). Roughly 50 percent of YouthBuild programs were concentrated in or near major metropol- 
itan areas with populations of a million or more (for example, New York City and Los Angeles) 
while about 20 percent were located in rural areas with populations of less than 50,000 (for 
example, Bemidji, Minnesota, and Hammond, Louisiana). These differing geographies have 
implications for recruitment, service delivery, and the availability of jobs. 

39 American Job Centers are the one-stop centers authorized by the Workforce Innovation and Opportunity 
Act of2014 to provide employment and training services to job seekers and workers. 
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Table 1.1 


Summary of Program Characteristics 


Program Response 

Percentage 
of Programs 

Percentage 
of Participants 

Type of orsanization 

Nonprofit 

82.7 

87.9 

National nonprofit 

12.0 

9.2 

Local/regional nonprofit 

70.7 

78.7 

Other 

17.3 

12.1 

Public agency 

10.7 

7.4 

Community college 

4.0 

3.3 

School district 

2.7 

1.5 

Geography 3 

Large metro, central 

34.7 

44.2 

Large metro, fringe 

17.3 

12.0 

Medium metro 

18.7 

24.2 

Small metro 

8.0 

2.3 

Rural 

21.3 

17.4 

Years in community 

Less than 20 

28.0 

35.8 

Between 20 and 50 

56.0 

53.5 

More than 50 

16.0 

10.8 

Years operating YouthBuild 

Less than 6 

24.0 

29.3 

Between 6 and 10 

36.0 

32.1 

More than 10 

40.0 

38.7 

YouthBuild USA affiliation status 

No affiliation status 

16.0 

15.5 

Provisional affiliate 

20.0 

24.1 

Full affiliate 

50.7 

43.8 

Accredited affiliate 

13.3 

16.7 

Sample size 

75 

3,929 


SOURCES: Calculations based on YouthBuild site-visit interview data, National Center for Health 
Statistics data, and affiliation information from YouthBuild USA. 

NOTE: “Geography is based on the National Center for Health Statistics scheme. For this table, 
micropolitan and noncore were combined to create the rural category listed here. The U.S. Virgin 
Islands were assigned a code based on total population and population density information from the 
2010 U.S. Census. Examples: Large metro, central: Atlanta, GA, and New York, NY; large metro, 
fringe: Gary, IN, and Tacoma, WA; medium metro: Spokane, WA, and Springfield, MA; small 
metro: Columbia, MO, and Jackson, MI; rural: Hammond, LA, and Traverse City, MI. 
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Figure 1.4 

Locations of Participating YouthBuild Programs 



SOURCE: YouthBuild study sample and program information. 


The ages of organizations operating YouthBuild also varied. They were generally well 
established, with a large majority having been in their communities for at least 20 years. The 
YouthBuild programs themselves ranged in age from 3 to 33 years, with an average of 10 years. 

Finally, some YouthBuild programs were affiliates of the YouthBuild USA Affiliated 
Network while others were not. Affiliate programs have access to additional funding and other 
types of program support, including AmeriCorps funding from CNCS and technical assistance. 
Three levels of affiliation — provisional, full, and accredited — are intended to describe a range 
of YouthBuild programs, from “provisional” status for programs in the planning and develop- 
ment phase to “accredited” status for well-established programs that meet YouthBuild USA 
performance standards and have demonstrated fidelity to the YouthBuild model. Nearly 85 
percent of programs in the evaluation were part of the YouthBuild USA network, and almost 65 
percent were full or accredited affiliates. 

Overall, the programs in the evaluation represent a wide variety of types. Some are in 
small towns and serve only a handful of young people each year, while others are in very large 
cities and enroll several cohorts per year. Some are just one program run by much larger 
umbrella organizations, while others are the main programs in their organizations, or even stand 
alone. This variety is also evident in the larger population of programs from which the evalua- 
tion programs were drawn. As shown in Appendix Table A.1, the programs participating in the 
study look very similar to this broader population of all programs that received funding from 
DOL or CNCS in 2011. 40 

Study Participants 

Table 1.2 presents selected characteristics of participating young people when they en- 
rolled in the study. The majority of participants are male (64 percent) and most are either black 
(63 percent) or Latino (15 percent). On average, study participants were nearly 20 years old 
when they entered the study, with about 70 percent older than age 18. 

About 10 percent of young people had a high school diploma or GED when they en- 
tered the study. Although the program targets young people who have dropped out of high 


40 A formal test of external validity was conducted for the DOL-funded programs, comparing the 60 pro- 
grams selected for the study with the broader sample of 74 DOL grantees. That test suggested that results from 
the DOL-funded program sample can be taken to represent (generalize to) the broader sample of grantees. 
However, a test was not conducted for the full sample of programs, since the group of CNCS-funded programs 
was not a random sample of programs receiving funding from CNCS. Nonetheless, as noted in the text, the 
study programs generally look similar to the broader population from which they were drawn. 


19 



Table 1.2 


Baseline Characteristics of the YouthBuild Sample 


Characteristic 

Percentage 
of Participants 

Age 

16-18 years old 

33.0 

19-21 years old 

46.3 

22 years old or older 

20.7 

Male (%) 

64.1 

Race/ethnicitv a 

Hispanic or Latino 

14.6 

White, non-Hispanic 

15.3 

Black, non-Hispanic 

62.9 

Other 11 

6.0 

Not specified 

1.1 

Has a child 

30.0 

Highest grade completed 0 

7th 

1.0 

8th 

7.5 

9th 

18.6 

10th 

26.2 

11th 

34.9 

12 th 

10.0 

Has a high school diploma or equivalent 

9.2 

Has a diagnosed disability (learning or physical) 

10.6 

Housing status 

Lives with family 

61.0 

Owns/rents apartment, room, or house 

15.2 

Is staying at someone’s apartment, room, or house 

15.7 

Is staying with foster guardian/in foster system 

0.6 

Lives in a halfway house/transitional house 

1.2 

Is homeless 

3.0 


(continued) 
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Table 1.2 (continued) 


Characteristic 

Percentage 
of Participants 

Who suggested you apply to YouthBuild? 

Family member or relative 

29.8 

No one 

32.5 

School counselor, truant officer, teacher, or principal 

4.3 

Friend 

20.7 

Other 

9.7 

Reasons for applying to YouthBuild 

GED 

87.7 

College 

63.1 

To get life on track 

88.2 

Job 

84.6 

Training 

67.2 

Friends 

7.0 

Because of children or the need to support family 

1.5 

Other 

4.4 

Sample size 

3,929 


SOURCE: Calculations based on the YouthBuild baseline data form. 

NOTES: a Categories are mutually exclusive. 

b "Other" includes Hawaiian Native or other Pacific Islander, Asian, American Indian or 
Alaskan, and responses of multiple races/ethnicities. 

c This information is missing for some sample members. 


school, funding guidelines allow programs to enroll some high school graduates . 41 Most of 
those who dropped out of high school did so after completing either tenth or eleventh grade, 
although about a third of them had completed ninth grade or less. As noted in the process study, 
many program staff members reported that participants typically had reading and math levels 
equivalent to middle school, even if they had completed some high school. 


41 Specifically, 25 percent of participants in programs funded by DOL can be young people who are not 
high school dropouts, who have high school diplomas, or who are not in one of the target groups, as long as 
they are deficient in basic skills or “have been referred by a local secondary school for participation ... leading 
to attainment of a secondary school diploma.” 
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Most young people reported that they decided on their own to enroll in YouthBuild or 
that they were referred to the program by family or friends. The most common reasons for 
applying to the program were to “get their life on track,” to obtain a GED, and to get a job. 
Fewer participants — though still more than half — listed “college” (63 percent) or “training” 
(67 percent) as reasons for applying. 


Findings from Other Youth Programs 

There have been a number of rigorous evaluations of programs for young people who have not 
completed high school. Overall, the programs tended to increase the rates at which participants 
received GEDs or other credentials, but they had less consistent effects on employment and 
earnings. To help place the YouthBuild evaluation and findings in this larger context, this 
section briefly discusses findings from several of these programs. The evaluation of Youth- 
Build, which also provides education and training services and which targets a similar popula- 
tion of young people as the programs discussed below, will add to the evidence base regarding 
what works to help disadvantaged young people. As did the evaluations listed below, because it 
is a randomized controlled trial the YouthBuild evaluation will provide the most rigorous level 
of evidence regarding the program’s effects. 

The JOBSTART Demonstration was an early attempt to test what might work for dis- 
advantaged young people. The programs in the demonstration were modeled on Job Corps 
(discussed below) but unlike Job Corps they offered their services in a nonresidential setting, as 
a way to test a less intensive and expensive model and one that might be available to a wider 
range of young people who might not be willing or able to live away from home while partici- 
pating. Programs in the demonstration operated at 13 sites in the mid-1980s, serving young 
people who had not completed high school and had low reading levels. The evaluation found 
that the program led to sizable increases in GED receipt (of 13 percentage points), but had few 
effects on work or earnings through four years. 42 The exception was that one program, the 
Center for Employment and Training (GET) in San Jose, California, produced relatively large 
increases in earnings. However, when the CET model was later replicated in 12 other locations, 
an evaluation found no increases in earnings through four and a half years. 43 

The American Conservation and Youth Service Corps (which, like YouthBuild, empha- 
sizes community service) offers participants temporary, full-time subsidized work in communi- 
ty service projects, along with case management, basic adult education and opportunities to earn 
college credits, and training in job-readiness skills. An impact evaluation in the late 1990s found 


42 Cave, Bos, Doolittle, and Toussaint (1993). 
43 Miller et al. (2005). 
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that more than a year after entering the study, Corps participants were more likely to have 
worked than control group members, and had also worked more hours. 44 However, a more 
recent and more comprehensive study of 2 1 Corps programs found no significant effects on 
educational attainment or work after 30 months. 45 

Both of these evaluations studied nonresidential programs, but some other, more inten- 
sive programs for disadvantaged young people — Job Corps and National Guard Youth 
ChalleNGe, for example — ask participants to leave their homes and live at the program site for 
a number of months. Job Corps, funded by DOL, is a national program serving disadvantaged 
young people ages 16 to 24. Young people participate in three phases of the program, first 
getting acclimated to the program’s culture and expectations, next participating in education and 
career-skills training (during which they receive modest stipends), and finally transitioning into 
employment and follow-up services after they go home. 

An evaluation of the program, launched in 1994, found that the average young person 
participated in the program for about eight months, and that it led to large increases in GED 
receipt and in the receipt of vocational certificates. For example, 42 percent of the program 
group earned a GED within four years after entering the study, compared with 27 percent of the 
control group. The program had no effects on college attendance. Job Corps is one of the few 
youth programs that led to positive effects on employment and earnings, although those effects 
faded after the fourth year of study follow-up. However, earnings gains did persist for older 
study participants (those who were ages 20 to 24 when they enrolled). 46 

National Guard Youth ChalleNGe serves a slightly younger population: 16- to 18-year- 
olds who have dropped out of high school. Participants engage in a five-month intensive 
residential program that includes eight core components: education, life skills, leadership skills, 
community service, citizenship building, physical fitness, health and hygiene, and job-skills 
training and career exploration. The program also helps participants set up structured mentoring 
for at least a year with mentors of their choice in their own communities, so that they have some 
support after they leave the program. A three-year study found that participants were more 
likely than their control group counterparts to have obtained high school credentials and to have 
earned college credits. 47 By Year 3, for example, 57 percent of the program group had received 
a GED, compared with 35 percent of the control group. The program also led to an increase in 
employment and earnings in Year 3. Longer-term follow-up is not available. See Appendix 
Table A.3 for more information on several evaluations of programs for young people. 


44 Jastrzab, Masker, Blomquist, and Orr (1996). 

45 Price et al. (201 1). 

46 Schochet, Burghardt, and McConnell (2008). 
47 Millenky, Bloom, Muller-Ravett, and Broadus (2011). 
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Organization of This Report 

The remainder of the report presents findings on participation in YouthBuild (among young 
people assigned to the program group) and program effects. Chapter 2 summarizes the findings 
from the process study on program implementation, presents data on participation in Youth- 
Build, and describes impacts on participation in education and training. Chapter 3 presents the 
program’s effects on a range of outcomes, including educational attainment, work and earnings, 
measures of youth development, and involvement in the criminal justice system. Chapter 4 
presents effects on important outcomes for selected subgroups, and Chapter 5 concludes. 
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Chapter 2 

Implementation and Participation 


This chapter discusses the program implementation findings, participating young people’s 
experiences with the YouthBuild program, and young people’s participation in YouthBuild and 
other services. The first section reviews findings from the earlier implementation report. The 
next section presents follow-up survey responses on young people’s participation in YouthBuild 
and level of satisfaction with the program. The third section presents impacts on participation in 
education, training, and other services. 


Implementation Findings 

In 2015, the first report of this evaluation was released, Adapting to Local Context, presenting 
implementation findings for the 75 participating YouthBuild programs. 1 Extensive implementa- 
tion data were collected and analyzed for that report, including multiday visits to each partici- 
pating program and an online survey of programs. 2 The participating programs were very 
diverse and were representative of the YouthBuild programs operating nationwide at the time. 
They varied in their geographic locations, the lengths of time they had been in their communi- 
ties, their organizational structures, and their funding and staffing arrangements. Overall, the 
evaluation found that programs were implementing the YouthBuild model faithfully. 

It is worth noting, however, that the YouthBuild program model described in Chapter 1 
is not highly prescriptive. Instead, it is designed to allow variation based on program and 
community contexts. For example, there was considerable variation in the fonnat of the educa- 
tional services offered by different YouthBuild programs. Yet even though the content and 
format of classes varied from program to program, most shared the goal of providing partici- 
pants individually tailored instruction and academic support. Small class sizes helped to 
promote positive relationships between instructors and participants. For vocational training, 
about a fifth of programs in the evaluation offered training in fields other than construction, 
such as health care, culinary arts, and computer technology. Training staff members reported 
often being stretched thin, and staff members overall reported that the economic downturn had 
significantly affected programs’ ability to implement construction-related vocational training. 
Specifically, it limited programs’ ability to provide quality construction experiences to partici- 
pants and limited the marketability of the job skills programs taught. Programs grappled with 


'Wiegand et al. (2015). 

2 Details of this “grantee survey” are discussed in Chapter 1 . 
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ways to diversify their operations through new partnerships and to provide new training 
opportunities for participants. 

Programs embraced the “culture” and value system of YouthBuild to varying degrees. 
This variation appeared most often in the leadership development component, a defining feature 
of YouthBuild, as noted in Chapter 1. While fidelity to the YouthBuild program model was 
high overall, not all programs were equally faithful in implementing the leadership development 
component. For example, a quarter of the participating programs did not have a functioning 
Youth Policy Council, a committee of students that plays an active role in setting decisions and 
policies that affect the program. 3 Programs that did not implement the leadership development 
component as faithfully as others often had fewer resources and less ability to dedicate staff 
time to these activities. 

Many programs adopt flexible staffing arrangements to deliver supportive, transition, 
and follow-up services. Programs often needed to distribute the delivery of these services across 
multiple staff members. For example, fewer than half of the programs had a designated job 
developer to give participants assistance with job searches and job placement. Programs also 
often combined life-skills and work-readiness training into one class, diluting their intensity but 
meeting the requirement to offer these services. 

YouthBuild programs typically were not alone in providing services to young adults in 
the communities where they operated. Although all the communities in this evaluation had other 
organizations offering some of the same services as YouthBuild, these alternative services 
rarefy matched the breadth and scope of those provided by YouthBuild. The services were 
usually not all available through the same provider. Therefore, it was possible for a young 
person to gain access to the same service components as provided by YouthBuild, but he or she 
would need to visit multiple providers to do so, and it seems unlikely that those providers would 
have coordinated their services. Similarly, few alternative programs in these communities 
seemed to create empowering enviromnents among their participants that could rival the 
environments developed by most of the YouthBuild programs in this study. 


Participation in YouthBuild 

Young people were asked about their participation in YouthBuild in the two follow-up surveys. 
Most participants had left the program before the 12-month survey. Because the 12-month 
survey took place closest to the time when young people would have received services, its data 


3 Programs must have an active Youth Policy Council in order to become members of YouthBuild USA’s 
Affiliated Network. However, it is not an element that the U.S. Department of Labor requires of its grantees. 
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are presented in Table 2.1. (Appendix Table C.l reports the corresponding data from the survey 
conducted at 30 months.) 

At 12 months, 74 percent of the program group reported that they had received 
YouthBuild services or participated in YouthBuild activities at some point in the past. This 
definition is broad, and does not necessarily indicate that a young person formally enrolled in 
YouthBuild. 4 As discussed in Chapter 1, in many cases random assignment occurred after 
Mental Toughness Orientation (MTO), so a survey respondent’s report of “participating” in 
YouthBuild might only mean that he or she had attended some part of MTO. However, these 
survey responses can be compared in some locations with the programs’ administrative data 
on participation. Among the 69 programs with available enrollment data (representing 87 
percent of the young people in the research sample), 73 percent of the program group formal- 
ly enrolled in YouthBuild (not shown in the table). 5 The fact that the participation rates shown 
in these two data sources are so similar suggests that a significant portion of the program 
group did participate in YouthBuild beyond MTO. It also means that a quarter of the program 
group participated in the YouthBuild application process and MTO at the most. As noted 
earlier, this report will present selected “impacts per participant” to adjust for this less-than- 
1 00-percent participation rate. 

Among those who reported receiving services, young people remained in YouthBuild 
for an average of eight months, and 50 percent reported completing the program. Once involved 
in program activities, most young people (87 percent) participated for more than three months. 
Again, these findings indicate that young people participated in YouthBuild beyond MTO and 
that the program staff members were able to engage them in the program model. In addition, 22 
percent of survey respondents said they were still enrolled in the program at the time of the 12- 
month survey. 

These participation levels are similar to those found in other evaluations of youth pro- 
grams. Job Corps’ program participation rate was also 73 percent, and young people reported 
staying in the program for an average of eight months. 6 In the National Guard Youth Chal- 
leNGe evaluation, 83 percent of the program group “registered” and began the residential pre- 
ChalleNGe orientation phase. Within that group, 64 percent completed the program. 7 


4 The timing of “enrollment” — and the definition of the term — varied among the 75 participating pro- 
grams, so it was not possible to ask in the survey interviews about formal enrollment into core YouthBuild 
services. 

5 Three programs did not enter the majority of their participants into either management information sys- 
tem available for this analysis. Due to their funding sources, these programs used their own management 
information systems. 

6 Schochet, Burghardt, and Glazerman (2001). 

7 Bloom, Gardenhire-Crooks, and Mandsager (2009). 
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Table 2.1 


YouthBuild Program Experience, Program Group Only, at 12 Months 

YouthBuild 


Survey Response Group 

Ever received YouthBuild services (%) 73.6 

Current status, among those who ever received YouthBuild services 

Currently enrolled 22.4 

Graduated from the program 49.6 

No longer enrolled and did not graduate 28.0 

Reasons for not participating in or not completing the program (%) 

Transportation 32.0 

Incarceration 10.2 

Program schedule 20.0 

Anotherjob 23.3 

Moved 19.7 

Birth of a child or child care problems 16.3 

Health issue or family pressure 19.1 

Conflict with the program, staff members, or other participants 21.7 

Expulsion or being asked to leave 19.4 

Another reason 27.2 

Among those who participated 

Months spent in YouthBuild 7.7 

In YouthBuild for more than 1 month (%) 95.6 

In YouthBuild for more than 3 months (%) 87.0 

Received a stipend from YouthBuild (%) 84.2 

Feel close to a YouthBuild staff person (%) 83.8 

Rated program, program component, or staff favorably 3 (%) 

Overall YouthBuild experience 87.4 

Construction or other job training 88.8 

Counseling 81.9 

Leadership training 86.8 

Help finding a job 68.7 

Help applying to college 77.7 

Understanding your needs 80.0 

Helping you solve problems 78.9 

Helping you learn 86.5 

Helping after you left YouthBuild 64.4 

Staying in contact after you graduated 75.5 


Sample size 1,830 


(continued) 
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Table 2.1 (continued) 


SOURCE: MDRC calculations based on responses to the 12-month survey. 

NOTE: “Indicates a response of "very good" or "good." The other response options were "okay," 
"poor," and "does not apply to me." 


Young people in the program group described a variety of reasons for either never at- 
tending the program or not completing it. Table 2.1 presents the reasons they provided at 12 
months. Thirty-two percent reported transportation issues and 23 percent had found another job. 
In the earlier implementation report, staff members also reported transportation as a major 
impediment to attendance. Other common reasons for not attending included family or health 
issues and a dislike of the program (the schedule, staff, or other participants). Notably, 10 
percent reported being incarcerated, and a slightly smaller percentage said that they stopped 
attending because the program closed (not shown). 8 

Overall, those who attended YouthBuild rated the services they received and the pro- 
gram’s staff favorably. Highly rated program services included the general program experience, 
the construction or job-training component, the counseling, and the leadership training. Similar- 
ly, 84 percent of participating young people reported feeling close to a YouthBuild staff person 
and rated the staff favorably on understanding participants’ needs and helping them leam. 

Although the ratings they gave in these areas were still fairly high, young people 
seemed slightly less satisfied with the services they received after they left the program — such 
as assistance finding a job or other forms of help — and with the staff members who delivered 
them. Some young people were in contact with the program relatively frequently after they left, 
while others were in contact relatively infrequently. Specifically, at 30 months young people 
were asked how frequently they were in contact with staff members from the program after they 
left. As shown in Table 2.2, 25 percent said they were in contact at least once a month and 
another 26 percent reported contact several times per year. The remaining half of the program 
group reported they were in contact once per year or not at all. Note that when young people 
were asked this question at 30 months, most had been out of YouthBuild for almost two years, 
which is well beyond the 12-month period for which programs receive funding to provide 
postprogram follow-up services. 


8 Program data from YouthBuild USA indicate that 13 programs closed after serving the cohorts that par- 
ticipated in the evaluation. Young people who said on the 12-month survey that their programs closed are 
probably not referring to those 13 programs, since those programs closed after serving the cohorts in the 
YouthBuild evaluation. 
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Table 2.2 


YouthBuild Program Experience, Program Group Only, at 30 Months 


Survey Response (%) 

YouthBuild 

Group 

Rated program favorably 3 

Caring staff 

90.5 

Safe and positive environment 

90.0 

Community service 

92.6 

Frequency of contact with YouthBuild staff members after leaving the program 

More than once a month 

9.7 

About once a month 

15.0 

A few times per year 

25.6 

Once per year 

13.6 

Not at all 

36.2 

Sample size 

1,811 


SOURCE: MDRC calculations based on responses to the 30-month survey. 

NOTE: indicates a response of "very good" or "good." The other response options were "okay," 
"poor," and "does not apply to me." 


These participant ratings echo the findings about transition and postprogram services 
presented in the implementation report. They also echo an earlier study that surveyed Youth- 
Build graduates and found that many of them wanted more contact with the program after 
graduation . 9 Most programs did not have staff members dedicated to cultivating employers, 
identifying job openings, and placing people in jobs. Smaller programs especially felt con- 
strained by funding and were unable to devote enough staff time to support young people with 
job searches and job placement. Staff members said it was sometimes challenging to follow up 
with participants after they left the program because their living situations and contact infor- 
mation changed frequently. Other challenges to follow-up included a lack of staff time and 
difficulty getting alumni to come to YouthBuild offices. 


9 Levine (2012). 
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Young people enrolled in the study and assigned to the control group were prohibited 
from enrolling in the participating YouthBuild program for two years. As noted earlier, these 
sample members were given information about other appropriate services in the community. An 
analysis of administrative program data shows that fewer than 1 percent of control group 
members formally enrolled in participating YouthBuild programs following random assignment 
(not shown). Note that this program information was only available for 69 programs, represent- 
ing 87 percent of the young people in the study sample. In addition, at the time of the evaluation 
there were 12 cities that were home to multiple YouthBuild programs, including programs not 
participating in the evaluation. Unfortunately, it is not possible to measure control group 
participation at YouthBuild programs not involved in the evaluation. About 18 percent of the 
young people in the control group did report on the 12-month survey that they had participated 
in YouthBuild. However, the question was asked in a very general way and could include a 
young person’s participation in the application process and Mental Toughness Orientation, in 
some cases. The fact that YouthBuild’s administrative data list fewer than 1 percent of control 
group members as enrolled in YouthBuild suggests that these control group members did not 
formally participate in the program. 


Impacts on Service Receipt 

The integration of education, vocational, and support services in the YouthBuild model suggests 
that young people in the program group should receive more and a wider range of services than 
their control group counterparts. As discussed in Chapter 1, these services, combined with the 
contextual factors of the program, the participants, and the community, can lead to positive 
outcomes for participating young people. However, young people who made it through the 
lengthy screening process were a motivated group at the time they applied. Therefore, the 
young people assigned to the control group are likely to have sought out alternative services on 
their own. This section examines the differences between the two groups in participation in 
educational, vocational, and other services. 

One year after random assignment, both research groups had participated in many ser- 
vices, but the program group had participated at statistically significantly higher rates in all 
types of services. 10 As shown in Table 2.3, services were divided into three domains that reflect 
the YouthBuild program model: education, job training, and personal development. Young 


10 As noted in Chapter 1, impacts are estimated using a regression framework in which the outcome of 
interest is regressed on an indicator variable for program status, plus additional variables measured at or before 
participants entered the study. Unless otherwise noted, all impacts discussed in the text are statistically 
significant at the 10 percent level or lower (see Appendix A for more detail). 
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Table 2.3 


Impacts on Service Receipt at 12 Months 



YouthBuild 

Control 

Difference 


Outcome 

Group 

Group 

(Impact) 

P-Value 

Education-related services 





Ever participated (%) 

75.0 

57.4 

yj 5 *** 

0.000 

GED preparation 

55.6 

37.9 

Yj 7 *** 

0.000 

Academic tutoring (not related to GED preparation) 

18.1 

7.6 

10.5*** 

0.000 

High school diploma prep courses 

25.5 

21.0 

4 5 *** 

0.007 

Standardized Achievement Test preparation 

25.0 

13.3 

1 1 g *** 

0.000 

College-preparation activities 3 

31.0 

11.6 

19 4 *** 

0.000 

Getting help finding financial aid 

30.9 

14.1 

7 *** 

0.000 

Other b 

11.0 

7.0 

4 o *** 

0.001 

Months participated 

4.6 

2.2 

2 4 *** 

0.000 

Job or training-related services 





Ever participated (%) 

70.8 

39.4 

314 *** 

0.000 

Job-skills training program 

43.2 

17.4 

25.8*** 

0.000 

On-the-job training in construction or another field 

54.6 

13.3 

41 3 *** 

0.000 

Job certification program 

31.1 

8.2 

22 9 *** 

0.000 

Job-search assistance 3 

54.2 

29.3 

24 9 *** 

0.000 

Help applying to a vocational training program 11 

35.1 

13.0 

22.1 *** 

0.000 

Months participated 

4.4 

1.4 

2 9 *** 

0.000 

Personal-development services 





Ever participated (%) 

59.3 

31.3 

28.0*** 

0.000 

Help or advice from a mentor 

38.5 

17.1 

2 j 3 *** 

0.000 

Life-skills training 3 

33.2 

10.1 

23 2 *** 

0.000 

Communication or public-speaking training 

31.0 

6.4 

24.6*** 

0.000 

Leadership-development training 

41.4 

8.7 

32.6*** 

0.000 

Health services 

27.9 

8.4 

19 ^ *** 

0.000 

Mental health services 

18.5 

6.0 

12 5 *** 

0.000 

Working with a case manager 

31.6 

12.9 

jg 2 *** 

0.000 

Months participated 

4.0 

1.7 

2 3 *** 

0.000 

Sample size (total = 2,845) 

1,852 

993 



(continued) 
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Table 2.3 (continued) 


SOURCE: MDRC calculations based on responses to the 12-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

“Includes college-awareness or college -guidance activities, college-preparation or -transition programs, 
and preparation for college entrance exams. 

b lncludes attending adult education classes, various certification courses, and college attendance. 
“Includes activities such as help filling out an application, writing a resume, and going for an interview, 
‘'includes help with a program application or interview. 

“Includes activities such as parenting-skills classes and learning how to balance a checkbook. 


people in the program and control group both reported receiving education-related services 
more often than services in the other two domains (75 percent of the program group and 57 
percent of the control group reported receiving education-related services). The one-year period 
following study enrollment roughly reflects the time when many young people in the program 
group were enrolled in YouthBuild and receiving its services. 

Within the education domain, General Educational Development (GED) test prepara- 
tion was the most common activity (reported by 56 percent of the program group and 38 percent 
of the control group). These GED preparation participation rates are strikingly similar to those 
for a similar period in the ChalleNGe evaluation. 1 1 The program group reported participating in 
a wide range of activities, while the control group members’ participation was less varied. Many 
young people did continue to work toward a high school diploma as well. On average, the 
program group reportedly spent more than four and a half months participating in education- 
related activities, while the control group spent about two months. 12 

The vast majority of the program group (76 percent of those who received services, not 
shown in the table) said that they participated in these services at YouthBuild. This finding is 
not surprising, since working toward a GED or a high school diploma is a central feature of the 
program. The YouthBuild model requires that academic services account for 50 to 60 percent of 
a YouthBuild participant’s time. For the control group, the most common setting was a school 
(32 percent). 


"in the ChalleNGe evaluation, 9 months after study enrollment (a span that covers the 5.5-month Chal- 
leNGe program period), 54 percent of the program group and 39 percent of the control group reported 
participating in GED preparation courses. See Bloom, Gardenhire-Crooks, and Mandsager (2009). 

12 For all the measures related to time spent on services in the three domains (discussed in this section and 
presented in Table 2.3), survey respondents were asked to report how much time they spent doing each 
individual activity within the domain. Since someone may have participated in many activities concurrently, 
Table 2.3 shows the averages of the longest times individuals reported for any activity within each domain. 


33 



Vocational training is another central component of the YouthBuild model. It is there- 
fore unsurprising that members of the program group were significantly more likely to partici- 
pate in job or training-related services than the control group (71 percent versus 39 percent). 
Again, the program group participated at high rates in a wide array of activities such as job- 
skills training and on-the-job training. The program group reported participating in these 
services for an average of three months longer than the control group. Seventy-eight percent of 
the program group members who received services reported receiving them at YouthBuild. The 
control group reported finding these services at community-based organizations, schools, 
employers, or elsewhere. 

In both groups, respondents reported participating in the personal-development services 
domain the least frequently of the three. However, nearly 60 percent of the program group 
participated in these types of services, compared with only 3 1 percent of the control group. 
Unsurprisingly, given YouthBuild’s focus on leadership, the largest difference between the 
program and control groups (33 percentage points) is in participation in leadership-development 
training. The program group also reported participating in each of the other activities more 
frequently than the control group. Seventy-eight percent of the program group members who 
received these services received them at a YouthBuild program. Most of those in the control 
group who participated in these services found them at community-based organizations and 
other types of organizations. 

At 30 months, the pattern of differences between the program group and the control 
group is similar to the pattern at 12 months, although higher percentages of both the program 
group and the control group reported receiving services in each domain than was the case at 12 
months, and the gaps between the program and control groups had narrowed. Figure 2.1 
presents a comparison between the 12- and 30-month surveys among young people who were 
interviewed at both times. See Appendix Table C.2 for a fuller accounting of the 30-month 
survey results. At 30 months, around 84 percent of the program group reported having partici- 
pated in education-related services (compared with 72 percent of the control group) and 8 1 
percent had participated in job or training-related services (compared with 59 percent of the 
control group). As a comparison, in the Job Corps evaluation, 30 months after study enrollment 
90 percent of program group members reported that they had enrolled in an education or 
training program at some time since random assignment, as did 64 percent of the control 
group. 13 


Overall, these results indicate that the YouthBuild group participated in services at 
higher rates than the control group, and participated in a wider variety of services. In addition, 


13 Schochet, Burghardt, and Glazerman (2000). 
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Figure 2.1 

Services Received According to the 12-Month and 30-Month Surveys 
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SOURCES: MDRC calculations based on responses to the 12- month and 30-month surveys. 
NOTE: Among those who responded to both surveys (sample size = 2,5 11). 


the control group participated in services in multiple locations. This finding reinforces those of 
the process study: Alternative service providers in these communities rarely offered the same 
breadth and depth of services as a YouthBuild program. However, the screening done by 
YouthBuild programs led to a highly motivated sample, so it is not surprising that participation 
rates for the control group were high and increased over time. 
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Chapter 3 

Impacts Through 30 Months 


As described in Chapter 2, young people in both the program group and the control group 
reported receiving a variety of services. However, young people in the program group partici- 
pated in services at significantly higher rates in all three domains measured: education, training, 
and personal development. Receiving these services is hypothesized to lead to better outcomes 
related to each of these domains. This chapter presents YouthBuild’s effects on education, 
employment, and youth development, as well as its effects in other areas, such as housing and 
involvement in the criminal justice system. 

As discussed in Chapter 1, data sources for these analyses include administrative rec- 
ords and the two follow-up surveys. Tables presented in this chapter reflect responses to the 30- 
month survey, when participating young people were 22 years old on average. See Appendix D 
for results from the 12-month survey. As also mentioned in Chapter 1, effects are occasionally 
discussed in terms of “impacts per participant” to account for the fact that not all young people 
assigned to the program group went on to participate in YouthBuild. Impacts per participant for 
selected outcomes are presented in Appendix Table D. 10. 

In sum, the program led to positive impacts on young people’s educational outcomes, 
particularly the rates at which they earned General Educational Development (GED) credentials 
and enrolled in vocational training and postsecondary education, as well as on some measures 
of employment and earnings. 1 There are few significant impacts on measures related to youth 
development and other outcome areas. 


Impacts on Education 

Educational services are essential components of the YouthBuild model: classes to help 
students complete their high school diplomas or GEDs; vocational services to prepare young 
people for careers; and services that help put students on a path to postsecondary education. As 
noted in Chapter 2, unless otherwise specified, only effects that are statistically significant at the 
10 percent level or lower are discussed as program impacts. See Appendix A for more detail on 
the methods used to estimate program impacts. 


'The term “GED” is used throughout the report to indicate a high school equivalency credential, even 
though many states no longer use the official GED test to grant those credentials. The official GED test was 
still the predominant test used, however, in the locations where sample members resided during the follow-up 
period. 
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Table 3.1 presents impacts on education and vocational training. Participants’ responses 
to the 30-month survey show that the program group was more likely than the control group to 
obtain a GED and receive vocational training or a trade license or certificate. Survey responses 
and enrollment data from the National Student Clearinghouse indicate that the program group 
was also more likely to enroll in postsecondary courses. At 30 months, 31 percent of the 
YouthBuild group had obtained a GED compared with only 18 percent of the control group, an 
impact of 14 percentage points. 2 The effect per participant is somewhat larger, at 19 percentage 
points. These findings are consistent with those reported in Chapter 2, showing that young 
people in the YouthBuild group were significantly more likely to receive GED preparation 
services than those in the control group. 3 Although a number of programs did offer services to 
help young people earn high school diplomas, there was no significant difference between the 
program and control groups in the percentages who earned high school diplomas. 

To provide two points of comparison, the impacts on GED receipt found here are simi- 
lar in size to those found in Job Corps but smaller than those in National Guard Youth Chal- 
leNGe. After 30 months, 34 percent of the program group in the Job Corps evaluation and 17 
percent of the control group had earned GEDs, rates similar to those presented here for Youth- 
Build. 4 After nearly two years, almost half of the program group in the ChalleNGe evaluation 
had earned a GED, compared with only 22 percent of the control group. 5 

The next two rows of Table 3.1 present data on enrollment in vocational training and 
the rate at which survey respondents earned trade licenses or certificates. Thirty-one percent of 
the YouthBuild group reported having enrolled in vocational training since they entered the 
study, compared with 20 percent of the control group, a statistically significant increase of 10 
percentage points. 6 Most of the increase in vocational training occurred during the program, 
although members of the YouthBuild group were also more likely to pursue training after they 


2 Data from the 2008 American Community Survey (ACS) indicate that about 21 percent of adults ages 20 
and older who dropped out of high school have GEDs. See Fry (2010). 

3 ln early 2014, the official GED test was modified to make it better reflect the skills young people need to 
be successful in college and the workplace. In addition, several states stopped using the GED and chose to use 
different tests to assess high school equivalency. Many have argued that the new official GED test is more 
difficult to pass, and test-taking rates fell dramatically between 2013 and 2014 (see Mulhere, 2015). As noted 
in Chapter 1, however, given the timing of the evaluation, most study participants would have taken the official 
GED test to earn their high school equivalency credentials. Nonetheless, the change to the test may have 
affected a small number of YouthBuild participants who took equivalency tests after they left the program. 

4 These numbers are among those who did not have the credential when they enrolled in the Job Corps 
evaluation. See Schochet, Burghardt, and Glazerman (2000). 

5 Millenky, Bloom, and Dillon (2010). 

6 Due to rounding there may appear to be a slight discrepancy in the figures shown here. 
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Table 3.1 


Impacts on Education and Training at 30 Months 


Outcome (%) 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Outcomes based on survey responses 

Earned a high school diploma or GED 

49.3 

36.7 

12.6*** 

0.000 

Eligh school diploma 

18.1 

19.2 

-1.1 

0.413 

GED 

31.2 

17.5 

7*H=H= 

0.000 

Enrolled in high school or GED classes 

65.3 

59.3 

6.0*** 

0.002 

Enrolled in vocational school 3 

30.8 

20.4 

10.3*** 

0.000 

Received a trade license/training certificate 13 

4.1 

2.1 

2.0** 

0.019 

Enrolled in postsecondary courses 

23.6 

18.1 

5.6*** 

0.001 

4-year college or university 

4.5 

2.2 

2 2 *** 

0.005 

2-year or community college 

22.0 

17.1 


0.004 

Received a postsecondary degree 

1.1 

1.0 

0.2 

0.680 

Associate's degree 

0.9 

0.5 

0.4 

0.300 

Bachelor's degree 

0.1 

0.1 

-0.1 

0.533 

Other degree 

0.3 

0.5 

-0.3 

0.254 

Sample size (total = 2,808) 

1,830 

978 



Outcomes based on enrollment data 

Attended college 

Enrolled in a 4-year institution 

3.0 

2.4 

0.6 

0.304 

Enrolled in a 2-year institution 

15.1 

8.0 

7 j*** 

0.000 

Enrolled in a less-than-2-year institution 

0.2 

0.0 

0.1 

0.302 

Public 

15.5 

8.0 

7 5*** 

0.000 

Private 

2.7 

2.4 

0.3 

0.572 

Full time 

7.6 

4.9 

2 7*** 

0.001 

Part time 

10.1 

3.9 

6.2*** 

0.000 

Received a degree 

1.4 

0.6 

0.8** 

0.032 

Certificate 

1.1 

0.4 

0 j** 

0.031 

Associate's 

0.2 

0.0 

0.2 

0.117 

Bachelor's 

0.1 

0.2 

-0.1 

0.457 

Master's 

0.0 

0.0 

0.0 

0.600 

Sample size (total = 3,929) 

2,700 

1,229 




(continued) 
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Table 3.1 (continued) 


SOURCES: MDRC calculations using enrollment data from the National Student Clearinghouse and 
responses to the 30-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
All outcomes reflect activity since random assignment unless otherwise noted. 

“"Vocational school" includes technical, business, and trade schools. 

b "Trade license/training certificate" includes technical, business, and trade certificates. 


left the program (based on analyses not shown). National data suggest that about 9 percent of 
GED recipients between the ages of 16 and 24 enroll in vocational or technical programs. 7 Both 
study groups are enrolling in these kinds of programs at a high rate, reinforcing the idea that the 
evaluation sample is a motivated group of young people. While few young people received 
trade licenses or training certificates (fewer than 5 percent of both groups), there was a statisti- 
cally significant difference between the two groups, with the program group about 2 percentage 
points more likely to complete a license or trade certificate. Note that the percentage of young 
people participating in vocational training reported here is somewhat lower than the reports of 
service receipt in general, presented through Month 12 in Chapter 2 and through Month 30 in 
Appendix Table C.2. This discrepancy most likely reflects the fact that the survey question used 
for the analysis in this chapter asked respondents about formal enrollment in vocational, 
technical, business, or trade school, not about participation in vocational training in general, 
which could include the training provided by YouthBuild itself. 

The next set of rows in the table presents effects on postsecondary enrollment. Given 
the positive effects on GED completion and many YouthBuild programs’ growing emphasis on 
postsecondary preparation, it is reasonable to expect effects on postsecondary enrollment. 
About 24 percent of young people in the program group reported having enrolled in postsec- 
ondary classes since random assigmnent, nearly 6 percentage points more than the control 
group. The effect per participant is somewhat larger, at 8 percentage points (see Appendix Table 
D.10). The survey data show that the YouthBuild group was more likely to report having 
enrolled in courses at both four-year and two-year colleges, although few members of either the 
program or the control group reported enrolling at four-year institutions. About 5 percent of the 
program group enrolled in a four-year college, compared with 2 percent of the control group. 
Enrollment rates for both groups are much higher at two-year colleges, and are 5 percentage 
points higher for the program group than they are for the control group. 


7 Zhang (2010). 
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Education outcomes are also reported using data from the National Student Clearing- 
house (NSC), which captures students’ enrollment at most postsecondary institutions in the 
United States, although its coverage is notably lower for for-profit institutions such as many 
technical schools. 8 Like the survey data, NSC data also show an increase in postsecondary 
enrollment, but the impact is only statistically significant for enrollment at two-year institutions. 
Fifteen percent of the YouthBuild group was enrolled at some point in a two-year program 
compared with 8 percent of the control group, an increase of 7 percentage points. The NSC data 
match the survey data in showing no effect on the rate at which young people earned associate’s 
degrees or higher degrees, but a small increase in the rate at which they earned certificates. The 
low rate of degree receipt is not surprising, since few young people would have had time to earn 
degrees during the 30-month follow-up period. 

To provide a point of comparison for these effects on postsecondary education, the ef- 
fect on certificate receipt is small compared with the effect found in the Job Corps evaluation. 
That program led to an increase in the receipt of vocational, technical, or trade certificates of 20 
percentage points, although it had no effect on the receipt of college degrees. 9 ChalleNGe, on 
the other hand, had no effect on certificate receipt (although vocational training was not part of 
its core services), but did lead to small increases in postsecondary enrollment. 10 

Data from the National Longitudinal Survey of Youth indicate that by the age of 22, 
about 8 percent of GED recipients have enrolled in two-year colleges and 2 percent have 
enrolled in four-year colleges. 11 These figures are comparable to the control group’s enrollment 
rates in this evaluation as measured using NSC data. The survey data show a somewhat higher 
rate of enrollment in two-year colleges, at 17 percent. 

These findings are encouraging given that YouthBuild has only recently begun to focus 
on postsecondary preparation. They are also encouraging given that the young people in this 
study (in both the program and control groups) appear to have been highly motivated to seek 
out and participate in services on their own. Although by the time of the 30-month survey few 
participants would have had time to complete a degree (and few of them did earn degrees), there 
is evidence to suggest that people who enroll in community college see higher earnings over 


8 GED recipients are more likely to enroll at for-profit colleges than students with traditional high school 
diplomas. Because so many members of the study sample are GED recipients, the NSC data may underrepre- 
sent the rates at which the program and control groups enrolled in college. See Dynarski, Hemelt, and Hyman 
(2015). 

9 Schochet, Burghardt, and Glazerman (2000). 

10 Millenky, Bloom, and Dillon (2010). 

1 'Calculations are based on the cross-sectional subsample of the survey, which is drawn to be representa- 
tive of the full population. See Heckman, Humphries, and Mader (2010). 
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time whether or not they ever earn degrees. 12 Further, some research suggests that additional 
years of schooling can improve health outcomes and reduce the rate at which people receive 
welfare benefits. 13 The final report from this evaluation will explore student persistence over a 
longer period. 


Impacts on Employment and Earnings 

The opportunities for education and training in YouthBuild should help participating young 
people to find jobs after completing the program. While few young people in the program group 
reported receiving trade licenses or certificates, Chapter 2 showed that their participation in 
training programs was relatively high. Table 3.2 presents YouthBuild’s effects on employment 
and earnings 30 months after study enrollment. (See Appendix D for 12-month impacts.) As 
Chapter 1 noted, this follow-up period ranges from 2012 through mid-2015, when youth 
unemployment rates were still quite high. As one recent paper notes, youth unemployment is 
more sensitive to the economic context than adult employment. 14 

While sample members were still quite young at the time of the survey (age 22, on av- 
erage) and some had enrolled in college, nearly 80 percent of the study sample reported having 
been employed at some point over the 30-month follow-up period, and just over 40 percent 
were employed at the time of the survey. Nearly a quarter reported full-time employment of 35 
hours or more a week, across one or more jobs. Although the program did not increase em- 
ployment rates, the program group was significantly more likely than the control group to be 
earning $10 or more per hour (18 percent of the program group reported earnings at that level 
compared with 14 percent of the control group, for an impact of 3 percentage points). 15 Con- 
sistent with this difference, the program also led to an increase in weekly earnings of $15.70. 
This difference in earnings is similar to the impact found in the Job Corps evaluation at the 
same 30-month follow-up point. Rates of employment for both the program and control groups 
were higher in Job Corps by about 10 percentage points. 16 The lower employment rates for the 
YouthBuild sample may be due in part to the weak youth labor market in the years following 
the Great Recession, as discussed in Chapter 1 . 


“Marcotte, Bailey, Borkoski, and Kienzl (2005). 

13 Belfield and Bailey (2011). 

14 Hossain and Bloom (2015). 

15 All wage and earnings averages include zeros for sample members who were not employed at the time 
of the survey. 

16 Schochet, Burghardt, and Glazerman (2000). 
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Table 3.2 


Impacts on Employment and Earnings at 30 Months 


Outcome 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Outcomes based on survey responses 

Ever employed since random assignment (%) 

80.2 

77.8 

2.4 

0.145 

Currently employed (%) 

44.3 

41.9 

2.4 

0.245 

Working full time (35+ hours/week) 

25.1 

24.7 

0.4 

0.820 

Self-employed 

11.1 

9.4 

1.7 

0.188 

Working through a temp agency 

7.1 

6.0 

1.1 

0.308 

Earning $ 10/hour or more 

17.5 

14.2 

3.3** 

0.030 

Average weekly earnings ($) 

150.2 

134.5 

15.7* 

0.081 

Current job industry (%) 

Construction 

4.2 

2.8 

1.4* 

0.075 

Retail trade 

6.3 

6.0 

0.3 

0.741 

Admin./support/waste mgmt./remediation 

5.6 

6.3 

-0.7 

0.484 

Health care and social assistance 

4.6 

4.0 

0.6 

0.474 

Accommodation and food service 

10.5 

10.9 

-0.3 

0.803 

Other 

12.3 

11.2 

1.1 

0.422 

Sample size (total = 2,808) 

1,830 

978 



Outcomes based on unemployment insurance data 

Ever employed (%) 

Employed in Year 1 

48.9 

51.3 

-2.4 

0.120 

Employed in Year 2 

62.9 

59.7 

3 2** 

0.049 

Employed in Quarter 10 

44.4 

44.8 

-0.4 

0.816 

Total earnings ($) 

Earnings in Year 1 

2,093.4 

2,407.7 

-314.3** 

0.016 

Earnings in Year 2 

3,735.2 

3,936.7 

-201.5 

0.293 

Earnings in Quarter 10 

1,280.8 

1,277.0 

3.9 

0.962 

Sample size (total = 3,877) 

2,662 

1,215 




SOURCES: MDRC calculations using data from the National Directory of New Hires and responses to the 30- 
month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Social Security numbers were not available for some sample members, who therefore could not be matched 
to the unemployment insurance database. 
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About 10 percent of the sample reported being self-employed (or just over 20 percent of 
those currently working). A separate analysis (not shown) indicates that the main difference in 
job types between the self-employed and other workers was that those reporting self- 
employment were more likely to work in construction-related jobs. 

The next set of rows in Table 3.2 presents the industry reported by sample members for 
their current jobs. Young people were most commonly employed in the accommodation and 
food service industry, a sector of the U.S. economy that employs many low-skilled workers. 
Rates of employment in the construction industry were quite low, especially when one consid- 
ers YouthBuild’s focus on construction. As discussed earlier, these low employment rates in 
construction could be linked to the fact that many of these young people lived in communities 
where construction had not rebounded from the recession. However, the program group was 
still more likely to be working in that field than the control group. 

Combining the survey responses with the administrative data, shown on the bottom of Table 3.2 
and in Figure 3.1, a pattern emerges related to program involvement: During the first year after 
study enrollment, when the program group was engaged in YouthBuild, the control group was 
more likely to be employed and earned more than the program group. After the program group 
left YouthBuild and perhaps obtained a GED or received other fonns of training, the program 
group’s employment increased and they were more likely to be employed than the control 
group. The administrative data show that the program group was more likely than the control 
group to be employed (63 percent versus 60 percent) in the second year of the follow-up period. 
Figure 3.1 illustrates the trend in earnings, with the control group earning significantly more in 
the first year, but the difference decreasing in the second follow-up year and the program group 
starting to earn more toward the end of the follow-up period (Quarter 9). This trend matches 
well with the survey data discussed above regarding current earnings and wages at the two-and- 
a-half-year mark. 


Impacts on Youth Development 

YouthBuild has been a leader in integrating youth development into its programs by promoting 
leadership and community service. These kinds of activities are intended to keep young people 
on a positive trajectory and increase their civic engagement. Several measures of youth devel- 
opment and civic engagement were obtained from the surveys. 

Table 3.3 presents the results. At 30 months, the program group was significantly more 
likely than the control group to report that they had volunteered or been involved in politics or 
local community activities. Fifty-four percent of the program group reported volunteering, 
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Figure 3.1 

Earnings by Quarter 
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SOURCE: MDRC calculations using data from the National Database of New Hires. 

NOTES: The earnings impacts are statistically significant at the 5 percent level in Quarters 2 and 4. The 
earnings impacts are statistically significant at the 1 percent level in Quarter 3 and at the 10 percent level in 
Quarter 7. 

Quarter 1 refers to the quarter of random assignment. This figure includes only employment and earnings 
in jobs covered by the unemployment insurance system. It does not include jobs not covered by the 
unemployment insurance system (for example, "off-the -books" jobs). 


compared with only 31 percent of the control group. Fifteen percent of the program group was 
involved in politics or local community activities, compared with 12 percent of the control 
group. Notably, both the program group and the control group volunteered at higher rates than 
the national average. Current Population Survey data from September 2012 indicate that only 
23 percent of those between the ages of 16 and 24 volunteer. 17 


17 U.S. Department of Labor, Bureau of Labor Statistics (2013). 


45 




Table 3.3 


Impacts on Youth Development at 30 Months 


Outcome 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Civic engagement since random assignment (%) 

92.2 

88.6 

3.6*** 

0.002 

Volunteered 

54.4 

30.9 

23.6*** 

0.000 

Registered to vote 2 

84.8 

84.7 

0.1 

0.965 

Voted 

46.4 

43.9 

2.4 

0.205 

Involved in politics or local community activities 

15.3 

12.0 

3.3** 

0.026 

Currently happy (%) 

80.2 

79.9 

0.4 

0.832 

What the future holds (%) 





Willing to wait for bigger financial rewards 11 

32.8 

36.8 

-4.0** 

0.047 

Will probably attend college 0 

74.3 

72.4 

1.9 


Expects to live at least 70 years 

82.0 

79.9 

2.1 

0.222 

Exhibits signs of major depression 0 ’ (%) 

16.7 

18.3 

-1.6 

0.325 

Overall good health (%) 

82.5 

82.7 

-0.3 

0.872 

Believes most people can be trusted (%) 

22.5 

20.9 

1.6 

0.364 

Social support score 0 

3.1 

3.1 

0.0 

0.421 

Self-esteem score* 

3.3 

3.3 

0.0 

0.545 

Self-confidence score 8 

3.0 

3.0 

0.0 

0.954 

Sample size (total = 2,808) 

1,830 

978 




SOURCES: MDRC calculations based on responses to the 30-month survey. 


NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

“Currently registered to vote at the time of the survey. 

b Based on responses to the question, "Would you rather get $80 tomorrow or get $100 three months from 
now?" 

“Among those who have not attended and are not currently attending college. This measure is 
nonexperimental, so significance level and P-Value are not included. 

depression is measured using the PHQ-9, a nine-item scale used to diagnose depression in clinical settings. 
Response categories range from 0 = "not at all" to 3 = "nearly every day," where higher scores indicate more 
frequent occurrence of depression symptoms. If the item score sum is greater than or equal to 10, the respondent 
is considered to exhibit signs of major depression. 

“Social support is measured using a six-item scale. Response categories range from 1 = "strongly disagree" to 4 
= "strongly agree," where higher scores indicate stronger social support. The six items are averaged. 

f Self-esteem is measured using the 10-item Rosenbserg Self-Esteem scale. Response categories range from 
1 = "strongly disagree" to 4 = "strongly agree," where higher scores indicate higher levels of self-esteem. The 10 
items are averaged. 

g Self-confidence is measured using a six-item scale. Response categories range from 1 = "strongly disagree" to 
4 = "strongly agree," where higher scores indicate higher levels of self-confidence. The six items are averaged. 
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The program had few other significant impacts related to youth development. The fol- 
low-up surveys asked a number of questions designed to capture aspects of development such 
as self-esteem, self-confidence, depression and happiness, and orientation toward the future. 
The program had no consistent or lasting effects on these measures. ChalleNGe, another 
program focused on promoting youth development, similarly had no effect on these types of 
outcome measures. 18 Other research suggests that it is difficult to create lasting changes in many 
of these attitudinal measures. 19 


Impacts on Other Outcomes 

The YouthBuild program model is designed to foster young people’s successful transition into 
adulthood, above and beyond their education and employment outcomes. To assess program 
impacts on outcomes in other important areas, the surveys asked respondents about their current 
living situations, family structures, substance use, and delinquency. Unless otherwise noted, the 
measures reported here are taken from the 30-month follow-up survey and either represent the 
status of respondents at that time or since the time of random assigmnent. (Data from the 12- 
month survey are presented in Appendix D.) 

Tables 3.4 and 3.5 present the results. Among the most notable outcomes in these ta- 
bles, young people in the program group were significantly more likely than control group 
members to report living on their own at the time of the survey, more likely to report living with 
their children (among those who had children), and less likely to report living in households that 
received government benefits. 

At the time of the 30-month survey, about half of the young people in both the program 
and control groups were living with their parents, and about 25 percent reported having been 
homeless at some point since random assignment. Young people in the program group were 
more likely to report that they lived on their own (30 percent compared with 26 percent of the 
control group), which seems to be mostly due to a reduction in living with parents. Perhaps as a 
result of this move, fewer young people in the program group were living in households whose 
members received government benefits (roughly 63 percent of the program group compared 
with 67 percent of the control group). 

The next several rows of Table 3.4 present data on parenthood status. Half of the study 
participants reported having children, up from 30 percent when they entered the study. Al- 
though the program did not affect the number of participants who had children, young people in 


Alillenky. Bloom, and Dillon (2010). 
19 Eccles and Gootman (2002). 
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Table 3.4 


Impacts on Living Arrangements and Household Information at 30 Months 


Outcome (%) 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Current living arrangement 





Parent's home 

45.8 

48.7 

-2.8 

0.152 

Another person's home 3 

18.8 

18.3 

0.5 

0.779 

One's own place 

29.6 

26.2 

3.4* 

0.062 

Incarceration facility 

1.9 

1.6 

0.4 

0.519 

Other arrangement 1 ’ 

3.9 

5.2 

-1.3 

0.115 

Ever homeless since random assignment 

23.9 

25.9 

-2.0 

0.252 

Married or living with spouse/partner 

26.1 

26.7 

-0.6 

0.737 

Receiving government benefits 0 

62.6 

66.6 

-40** 

0.041 

Has children 11 

51.3 

49.1 

2.2 

0.187 

Young parent 3 

1.7 

2.3 

-0.6 

0.266 

Lives with all or some of one's children 

37.4 

33.8 

3.6** 

0.033 

Has children but not custody 

18.2 

18.5 

-0.3 

0.828 

Paid child support in the last 30 days 

3.7 

2.9 

0.8 

0.268 

Sample size (total = 2,808) 

1,830 

978 




SOURCE: MDRC calculations based on responses to the 30-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
“Includes living with family other than parents. 

b lncludes living in a group home or halfway house, a long-term homeless shelter, or an emergency 
housing shelter (including for domestic violence); living on the street; situations such as college or 
residential training programs; and other situations. 

“Government benefits include Temporary Assistance for Needy Families; the Supplemental Nutrition 
Assistance Program (food stamps); unemployment insurance; the Special Supplemental Nutrition Program 
for Women, Infants, and Children; Supplemental Security Income; foster care payments; and utility- 
payment assistance. 

includes a person's biological, adopted, foster, and stepchildren, plus any other children he or she is 
responsible for. 

“"Young parent" is defined here as a person under the age of 20 who is pregnant or who has a child. 
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Table 3.5 


Impacts on Delinquency and Risky Behavior at 30 Months 


Outcome (%) 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Arrested 

27.6 

26.4 

1.3 

0.482 

Charged 

24.8 

23.7 

1.1 

0.526 

Convicted or found delinquent 3 

15.7 

14.1 

1.6 

0.278 

Drag offense 

4.8 

4.3 

0.5 

0.543 

Driving under the influence 

1.3 

1.2 

0.1 

0.897 

Failure to pay child support 

0.2 

0.1 

0.1 

0.568 

Property offense b 

5.2 

4.7 

0.4 

0.656 

Violent offense 0 

3.7 

2.7 

1.0 

0.186 

Other 

2.5 

1.3 

1.3 ** 

0.037 

Locked up due to a sentence 

8.9 

7.3 

1.6 

0.176 

Involved in a gang fight in the past 12 months 

5.6 

6.7 

-1.0 

0.297 

Substance abuse 

Has 5+ drinks once or more in a typical week 

31.7 

26.1 

5.6*** 

0.004 

Has used marijuana since random assignment 

45.8 

48.2 

-2.5 

0.229 

Has used another drug since random assignment 
Drove a car while drinking or doing drugs 

13.4 

9.7 

3.6*** 

0.009 

in the last 30 days 

6.0 

4.5 

1.5 

0.118 

Sample size (total = 2,808) 

1,830 

978 




SOURCES: MDRC calculations based on responses to the 30-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

All outcomes reflect activity since random assignment unless otherwise noted. 

Tn the juvenile justice system, the term "adjudicated delinquent" is used rather than "convicted." 
b Property offenses include shoplifting, burglary, larceny, theft, auto theft, writing bad checks, fraud, forgery, 
arson, vandalism, and possession of stolen goods. 

c Violent offenses include physical or sexual assault, rape, robbery, manslaughter, attempted murder, and 
murder. 
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the program group were somewhat more likely than those in the control group to report that 
they lived with all or some of their children (37 percent compared with 34 percent). This 
difference does not appear to be caused by a reduction in the number of noncustodial parents, 
since there is no effect on that outcome. 

The survey also asked questions about delinquency and about risky behaviors such as 
substance use. The first few rows of Table 3.5 present effects on involvement in the criminal 
justice system. About 27 percent of the young people in the study reported having been arrested 
since random assigmnent, and 15 percent had been convicted. Most of these convictions were 
for drug or property offenses, which is in line with national data on common offense types for 
young adults. 20 The program had no effect on these outcomes. Rates of arrests reported by the 
YouthBuild sample are similar to those found in the Job Corps evaluation. In that study, 27 
percent of young people in the control group reported having been arrested in the 30 months 
after they entered the study. Job Corps led to a reduction in that rate of 4 percentage points. 21 

The bottom panel of Table 3.5 shows rates of substance use. At the time of the 30- 
month survey, young people in the program group were more likely than those in the control 
group to report periods of heavy alcohol use (32 percent versus 26 percent, for an impact of 6 
percentage points). Although it is not clear what led to this finding, it is cause for concern, and a 
future report will assess whether it persists over time. (There was no effect on this outcome at 
12 months.) In contrast, there was no significant difference between the program and control 
groups in the use of marijuana (roughly 47 percent of respondents had used marijuana at least 
once). As a point of comparison, in a national survey about 20 percent of young adults reported 
using marijuana in the previous month. 22 The rate reported by the study sample is expected to be 
higher because it covers a 30-month period. Finally, program group members were significantly 
more likely than control group members to report having used drugs other than marijuana (13 
percent compared with 10 percent). 


20 Sickmund and Puzzanchera (2014). 

21 Schochet, Burghardt, and Glazerman (2000). 

22 U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Admin- 
istration (2014). 
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Chapter 4 

Impacts on Subgroups 


Although the young people who enrolled in the study are all disadvantaged and share many 
common challenges, they do differ in many ways. Some study participants were in their 20s 
when they entered the study, while others were still teenagers. Some nearly finished high school 
before dropping out, while others left school in the ninth or tenth grade. Still others are parents 
of young children. YouthBuild might have different effects on the different types of young 
people it serves. As the process study noted, for example, many staff members believe that older 
participants were more ready to take advantage of the program. This chapter presents effects for 
a limited number of preselected subgroups of young people. 

The process study also found that although most programs had high fidelity to the 
YouthBuild model, there was some variation among them, particularly in the youth- 
development component. This chapter assesses how program impacts vary based on overall 
program fidelity. (The programs in the study also vary in many other ways, such as the length of 
Mental Toughness Orientation, the strength of their connections to colleges, and their emphasis 
on youth leadership. A fomial analysis of how YouthBuild’s effects vary across some of these 
program dimensions will be presented in the final report.) 


Selection of Subgroups 

The subgroups based on young people’s characteristics were defined by age, sex, and 
education level. The selection of subgroups was informed by underlying theory or previous 
evidence about how certain characteristics might affect how young people interact with or 
benefit from the program. Age, for example, was an important dimension because staff mem- 
bers observed that older participants tend to be more ready for the program. In addition, the Job 
Corps evaluation found that the program’s effects on earnings persisted longer for those who 
were older when they entered the study. 1 On the other hand, older participants would have been 
out of school longer and may have found it more challenging to engage in an education pro- 
gram. “Younger” participants were defined as those who were under age 20 when they entered 
the study, while “older” participants were defined as those 20 and older. 


'Schochet. Burghardt, and McConnell (2008). 
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The program might also have different effects on men and women. The challenges fac- 
ing young black and Latino men have been well documented and are evidenced by their high 
dropout rates and unemployment rates. 2 In this sense, young men stand to benefit the most from 
a program like YouthBuild and might see bigger effects than young women. On the other hand, 
the challenges they face may limit their ability to take advantage of the program. Other evalua- 
tions of youth programs have found that they have different effects on young men and women. 3 

The final subgroup is defined by education level, or the highest grade a young person 
completed before leaving high school. As discussed in the process report, staff members noted 
that many young people entered the study with middle school reading and math levels, and a 
number of programs use assessments of basic skills to screen out very low-skilled candidates. 
On the one hand, young people with more education may be in a better position to take ad- 
vantage of YouthBuild’s services, passing the General Educational Development (GED) exam 
and benefiting from vocational training. 4 On the other hand, young people with more education 
might have done fairly well even in the absence of YouthBuild (especially since this appears to 
have been a motivated sample of young people), so there might not be much difference between 
the program and control groups’ outcomes. The “less educated” subgroup of participants are 
those who left school before completing the tenth grade, while the “more educated” subgroup 
are those who completed the tenth grade or more. Note that the “more educated” subgroup 
includes the 9 percent of sample members who had high school diplomas or GEDs when they 
entered the study. 

The final subgroup is based on a program characteristic: fidelity to the YouthBuild 
model. The process study documented that most programs had high fidelity to the model. The 
average fidelity score was 79 out of 100. Overall fidelity was based on scores for each of the 
following components: educational services, vocational training, supportive services, program 
operations, youth leadership, and postsecondary preparation. Average fidelity scores were 
highest for education and vocational training and somewhat lower for youth leadership and 
postsecondary preparation. The general score is used here to provide an initial look at how 
impacts vary with the overall strength of the program. Since the average fidelity score was 79 
for study programs, high-fidelity programs were defined here as those achieving overall scores 
of 80 or higher. About half of the programs are above this cutoff. 

The analysis is conducted by estimating impacts separately for each of the two sub- 
groups in a pair and then assessing the differences in effects. In general, impacts are expected to 

2 U.S. Department of Education, National Center for Education Statistics (2014); U.S. Department of La- 
bor, Bureau of Labor Statistics (2016d). 

3 See, for example, Miller et al. (2005). 

4 As noted earlier, the term “GED” is used throughout this report to indicate a high school equivalency 
credential, even though many states no longer use the official GED test to grant those credentials. 
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vary to some extent between subgroups, simply as a result of natural variation around the 
average impact for the full sample. The analysis assesses whether that variation in impacts 
across subgroups is statistically significant, or beyond what would be expected to occur by 
chance alone. For that reason, the important question is not whether a given impact for, say, the 
younger subgroup is statistically different from zero, but whether that impact is statistically 
different from the impact for the older subgroup (indicated by daggers in the rightmost column 
of the tables). If the difference between these two subgroup impacts is not statistically signifi- 
cant, the results suggest that the effects observed for the full sample generally hold across both 
groups being compared. 


Subgroup Impacts 

Table 4.1 presents YouthBuild’s effects on the younger and older subgroups of participants. 
Both groups have similar rates of participation in YouthBuild, although the older group is 
somewhat more likely to have graduated from the program. The only difference in impacts 
between the groups is a large increase in GED receipt for the older group. The program in- 
creased GED receipt by 17 percentage points for the older group, compared with 1 1 percentage 
points for the younger group. The relatively large increase in GED receipt for the older group is 
encouraging. Older participants may find it more difficult to reengage in education, since they 
have been out of school for a longer time — as reflected in the responses of the control group, 
which show lower rates of participation in education and GED receipt for the older subgroup. 

Table 4.2 presents the program’s effects on women and men. Women and men were 
equally likely to graduate from YouthBuild. In fact, overall, the program had similar effects on 
men and women. One important difference between women and men does not relate to program 
effects, but to the general outcomes of both groups: Women had much lower rates of involve- 
ment with the criminal justice system. About 15 percent of women in the control group reported 
having been arrested since they entered the study, compared with 34 percent of men. 

Table 4.3 presents YouthBuild’s effects on the more and less educated subgroups. The 
more educated group is more likely to have graduated from the program, although Youth- 
Build’s impacts on service receipt and GED receipt are very similar for both groups. The more 
educated control group was less likely to have earned a GED in the time since they entered the 
study (probably because more of them already had GEDs or diplomas when they entered), but 
much more likely to have enrolled in postsecondary education. 

The only difference in effects across the two groups is for work. The program led to an 
increase of 6 percentage points in work as reported on the survey for the less educated sub- 
group, an effect that is significantly different from the effect on the more educated subgroup. 
The effects on work found using unemployment insurance data are also significantly different 
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between the two subgroups, although that difference is due in part to a negative effect on 
employment for the more educated group. It will be important to track longer-term effects for 
both of these groups. 

Table 4.4 presents the effects of higher-fidelity and lower- fidelity programs. Effects on 
service receipt are larger for young people in the higher-fidelity programs. For example, the 
higher-fidelity programs increased participation in job- or training-related services by 26 
percentage points, compared with 14 percentage points for the lower-fidelity programs. These 
impacts on service receipt seem to have arisen in part because program group members at 
higher-fidelity programs engaged in services at higher rates, but also because the control groups 
at these programs engaged in services at lower rates. Other differences between the control 
groups similarly suggest that the higher-fidelity programs may have served more highly 
disadvantaged young people. For example, the control group for the higher-fidelity programs is 
less likely than the other control group to have earned GEDs, and less likely to have attended 
college. In the final report, the analysis of how effects vary among programs will account for 
differences in the types of young people they serve. In any case, the results shown here indicate 
that the programs that had higher fidelity to the YouthBuild model did not have larger effects. 
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Table 4.1 


Impacts by Age at 30 Months 



Sample Members Under 20 


Sample Members 20 and Older 



Outcome 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Difference 

Between 

Subgroup 

Impacts 

Ever participated in YouthBuild (%) 

80.0 

NA 

NA 


79.9 

NA 

NA 



Graduated from YouthBuild (%) 

65.6 

NA 

NA 


70.5 

NA 

NA 



Service receipt (%) 










Education-related 

82.4 

72.1 

10.3*** 

0.000 

79.9 

67.4 

12 5*H=H= 

0.000 


Job- or training-related 

78.3 

59.2 

jp jh=h=* 

0.000 

78.3 

56.2 

22.1*** 

0.000 


Personal development 

65.0 

44.5 

20.5*** 

0.000 

66.2 

44.2 

22.0*** 

0.000 


Education (%) 










Earned a GED since random assignment 
Enrolled in college since random 

30.5 

19.8 

7*H=H= 

0.000 

32.2 

15.2 

jy QH=H=H= 

0.000 

t 

assignment (2- or 4-year) 

22.7 

18.4 

4.3* 

0.081 

24.6 

17.8 

6.8*** 

0.005 


Youth development 










Civic engagement 3 (%) 

90.9 

87.4 

3.5** 

0.050 

93.5 

89.7 

3.8** 

0.012 


Self-esteem scale b 

3.3 

3.2 

0.1* 

0.071 

3.3 

3.3 

0.0 

0.704 


Work 










Currently employed (%) 

43.2 

40.6 

2.6 

0.384 

45.1 

43.5 

1.6 

0.584 


Earnings greater than $ 1 0 per hour (%) 

16.5 

12.5 

4.0* 

0.064 

18.7 

15.5 

3.2 

0.153 


Ever employed in Quarter 10 c (%) 

44.0 

42.0 

2.0 

0.411 

44.9 

47.6 

-2.7 

0.249 



(continued) 



Table 4.1 (continued) 


Sample Members Under 20 Sample Members 20 and Older 

Difference 

Between 

YouthBuild Control Difference YouthBuild Control Difference Subgroup 


Outcome 

Group 

Group 

(Impact) 

P-Value 

Group 

Group 

(Impact) 

P-Value 

Impacts 

Work 

Average earnings in Quarter 10 c ($) 

1,266 

1,197 

68.3 

0.537 

1,301 

1,343 

-41.7 

0.700 


Involvement in the criminal justice system 
Arrested since random assigment (%) 

28.9 

28.9 

0.0 

0.994 

26.0 

24.5 

1.5 

0.555 


Sample size 

950 

467 



879 

510 





SOURCES: MDRC calculations using data from the National Directory of New Hires and responses to the 30-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The H-statistic is used to assess whether the difference between subgroup impacts is statistically significant. Statistically significant differences between 
subgroups are indicated as: fff = 1 percent; ff = 5 percent; f = 10 percent. 

NA = not applicable. 

a "Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, and being involved in politics or local community 
services. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 1 = "strongly disagree" to 4 = "strongly agree," 
where higher scores indicate higher levels of self-esteem. Responses to the 10 items are averaged. 

c These outcomes are based on unemployment insurance data, not survey responses. The sample sizes for the younger program and control subgroups are 1,414 
and 583 respectively. The sample sizes for the older program and control subgroups are 1,248 and 633 respectively. 



Table 4.2 


Impacts by Gender at 30 Months 


Outcome 


Women 



Men 


Difference 

Between 

Subgroup 

Impacts 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Ever participated in YouthBuild (%) 

78.9 

NA 

NA 


80.6 

NA 

NA 



Graduated from YouthBuild (%) 

67.6 

NA 

NA 


68.2 

NA 

NA 



Service receipt (%) 










Education-related 

82.3 

74.1 

g 2*** 

0.003 

80.6 

67.0 

13.6*** 

0.000 


Job- or training-related 

76.5 

56.5 


0.000 

79.5 

58.4 

21 

0.000 


Personal development 

67.6 

47.1 

20.5*** 

0.000 

64.5 

42.3 

22.2*** 

0.000 


Education (%) 










Earned a GED since random assignment 

26.6 

16.4 

10.2*** 

0.000 

33.9 

18.4 

15.5*** 

0.000 


Enrolled in college since random 










assignment (2- or 4- year) 

24.3 

20.6 

3.7 

0.215 

22.7 

17.4 

5.4** 

0.012 


Youth development 










Civic engagement 3 (%) 

93.2 

88.4 

4 g*** 

0.010 

91.6 

88.7 

2.8* 

0.062 


Self-esteem scale * 5 

3.2 

3.3 

0.0 

0.763 

3.3 

3.3 

0.0 

0.201 


Work 










Currently employed (%) 

39.8 

38.3 

1.4 

0.675 

47.4 

43.7 

3.6 

0.168 


Earnings greater than $ 10 per hour (%) 

12.0 

7.7 

44 ** 

0.044 

21.5 

17.4 

4.1* 

0.060 


Ever employed in Quarter 10 c (%) 

45.2 

43.9 

1.3 

0.660 

44.1 

45.4 

-1.3 

0.551 



(continued) 



Table 4.2 (continued) 




Women 



Men 




Outcome 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Difference 

Between 

Subgroup 

Impacts 

Work 

Average earnings in Quarter 10 c ($) 

1,100 

1,026 

73.4 

0.483 

1,383 

1,418 

-35.2 

0.756 


Involvement in the criminal justice system 
Arrested since random assignment (%) 

16.5 

14.8 

1.7 

0.509 

34.2 

34.5 

-0.3522 

0.89 


Sample size 

697 

366 



1,127 

608 





SOURCES: MDRC calculations using data from the National Directory of New Hires and responses to the 30-month survey. 


NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The H-statistic is used to assess whether the difference between the subgroup impacts is statistically significant. Statistically significant differences between 
subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

NA = not applicable. 

a "Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, and being involved in politics or local community 
services. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 1 = "strongly disagree" to 4 = "strongly agree," 
where higher scores indicate higher levels of self-esteem. Responses to the 10 items are averaged. 

c These outcomes are based on unemployment insurance data, not survey responses. The sample sizes for the female program and control subgroups are 955 
and 424 respectively. The sample sizes for the male program and control subgroups are 1,703 and 789 respectively. 



Table 4.3 


Impacts by Highest Grade Previously Completed, at 30 Months 


Sample Members Who Had Completed Sample Members Who Had Completed 

Less Than Tenth Grade Tenth Grade or Higher 

Difference 

Between 

YouthBuild Control Difference YouthBuild Control Difference Subgroup 


Outcome 

Group 

Group 

(Impact) 

P- Value 

Group 

Group 

(Impact) 

P-Value 

Impacts 

Ever participated in YouthBuild (%) 

80.7 

NA 

NA 


79.1 

NA 

NA 



Graduated from YouthBuild (%) 

65.0 

NA 

NA 


71.5 

NA 

NA 



Service receipt (%) 










Education-related 

83.4 

71.0 

Y2 4 *** 

0.000 

79.7 

68.7 

J J QH=H=H= 

0.000 


Job- or training-related 

79.5 

56.9 

22 . 6 *** 

0.000 

76.8 

58.5 

18.3*** 

0.000 


Personal development 

68.0 

45.0 

23 0 *** 

0.000 

62.9 

43.6 

19 2 *** 

0.000 


Education (%) 










Earned a GED since random assignment 

32.3 

18.2 

j4 j*** 

0.000 

29.8 

15.1 

14 7*** 

0.000 


Enrolled in college since random 










assignment (2- or 4- year) 

20.8 

13.9 


0.002 

27.0 

22.6 

4.4 

0.106 


Youth development 










Civic engagement 3 (%) 

91.1 

86.9 

4.2** 

0.012 

93.4 

90.7 

2.7* 

0.096 


Self-esteem scale b 

3.3 

3.3 

0.0 

0.634 

3.3 

3.3 

0.0 

0.436 


Work 










Currently employed (%) 

44.2 

37.9 

6.3** 

0.028 

44.0 

46.7 

-2.7 

0.380 

ft 

Earnings greater than $10 per hour (%) 

16.6 

12.2 

4 4 ** 

0.032 

18.3 

16.9 

1.3 

0.583 


Ever employed in Quarter 10 c (%) 

43.3 

40.3 

3.0 

0.190 

45.7 

50.2 

-4.6* 

0.071 

ft 


(continued) 



Table 4.3 (continued) 


Sample Members Who Had Completed Sample Members Who Had Completed 

Less Than Tenth Grade Tenth Grade or Higher 


Outcome 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Difference 

Between 

Subgroup 

Impacts 

Work 

Average earnings in Quarter 10 c ($) 

1,190 

1,094 

96.1 

0.295 

1,399 

1,494 

-95.7 

0.493 


Involvement in the criminal justice system 
Arrested since random assignment (%) 

29.5 

27.4 

2.1 

0.412 

25.1 

26.0 

-0.9 

0.738 


Sample size 

1,004 

515 



797 

450 





SOURCES: MDRC calculations using data from the National Directory of New Hires and responses to the 30-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The H-statistic is used to assess whether the difference between the subgroup impacts is statistically significant. Statistically significant differences between 
subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

NA = not applicable. 

a "Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, and being involved in politics or local community 
services. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 1 = "strongly disagree" to 4 = "strongly agree," 
where higher scores indicate higher levels of self-esteem. Responses to the 10 items are averaged. 

"These outcomes are based on unemployment insurance data, not survey responses. The sample sizes for program and control subgroup members who had 
completed less than tenth grade are 1,445 and 631 respectively. The sample sizes for program and control subgroup members who had completed tenth grade or 
higher are 1,177 and 571 respectively. 



Table 4.4 


Impacts by Program Fidelity, at 30 Months 

Sample Members from Programs with Sample Members from Programs with 

Low Fidelity High Fidelity 

Difference 

Between 


Outcome 

YouthBuild 

Group 

Control Difference 
Group (Impact) 

P- Value 

YouthBuild 

Group 

Control Difference 
Group (Impact) 

P- Value 

Subgroup 

Impacts 

Ever participated in YouthBuild (%) 

75.7 

NA 

NA 


82.7 

NA 

NA 



Graduated from YouthBuild (%) 

69.0 

NA 

NA 


67.3 

NA 

NA 



Service receipt (%) 










Education-related 

80.2 

72.3 

7 g *** 

0.003 

81.9 

67.7 

14 0 *** 

0.000 

t 

Job- or training-related 

75.9 

62.4 

13 5 *** 

0.000 

79.8 

54.1 

25.8*** 

0.000 

ftf 

Personal development 

63.0 

46.0 

yj o*** 

0.000 

67.3 

42.9 

24 4 *** 

0.000 

f 

Education (%) 










Earned a GED since random assignment 
Enrolled in college since random 

33.8 

19.1 

24 7 *** 

0.000 

29.4 

16.8 

12 . 6 *** 

0.000 


assignment (2- or 4- year) 

27.4 

21.0 

6.3** 

0.022 

21.1 

16.3 

4g** 

0.026 


Youth development 










Civic engagement 3 (%) 

93.2 

90.3 

2 . 8 * 

0.095 

91.5 

87.3 

4 3 *** 

0.006 


Self-esteem scale b 

3.3 

3.3 

0.0 

0.855 

3.3 

3.3 

0.0 

0.590 


Work 










Currently employed (%) 

44.0 

43.6 

0.5 

0.881 

44.5 

40.4 

4.1 

0.132 


Earnings greater than $10 per hour (%) 

18.1 

16.6 

1.6 

0.522 

17.1 

12.5 

4.6** 

0.021 


Ever employed in Quarter 10 c (%) 

42.5 

46.0 

-3.5 

0.166 

45.8 

43.8 

2.0 

0.373 



(continued) 



Table 4.4 (continued) 


Sample Members from Programs with Sample Members from Programs with 

Low Fidelity High Fidelity 

Difference 

Between 

Program Control Difference Program Control Difference Subgroup 


Outcome 

Group 

Group 

(Impact) 

P- Value 

Group 

Group 

(Impact) 

P- Value 

Impacts 

Work 

Average earnings in Quarter 10 c ($) 

1,232 

1,257 

-25.3 

0.824 

1,315 

1,290 

25.7 

0.804 


Involvement in the criminal justice system 
Arrested since random assignment (%) 

26.3 

24.5 

1.8 

0.520 

28.6 

27.6 

1.0 

0.692 


Sample size 

703 

425 



1,127 

553 





SOURCES: MDRC calculations using data from the National Directory of New Hires and responses to the 30-month survey. 


NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The H-statistic is used to assess whether the difference between the subgroup impacts is statistically significant. Statistically significant differences between 
subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

NA = not applicable. 

a "Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, and being involved in politics or local community 
services. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 1 = "strongly disagree" to 4 = "strongly agree," 
where higher scores indicate higher levels of self-esteem. Responses to the 10 items are averaged. 

c These outcomes are based on unemployment insurance data, not survey responses. The sample sizes for the low-fidelity program and control subgroups are 
1,076 and 540 respectively. The sample sizes for the high-fidelity program and control subgroups are 1,586 and 676 respectively. 



Chapter 5 

Conclusion 


YouthBuild attempts to engage disadvantaged young people who have not completed high 
school and help them make a successful transition to adulthood. It does so by offering educa- 
tion, job training, and other services in a program environment that emphasizes youth develop- 
ment and leadership. This report has presented the interim effects of the program, tracking 
young people for 30 months after they entered the study, or about 18 to 20 months after most 
participants left the program. 

The results show a number of positive effects, most consistently on education and train- 
ing. YouthBuild increased the rate at which young people earned their General Educational 
Development (GED) credentials and participated in vocational training, and led to an increase in 
postsecondary enrollment. Effects on work and earnings were less robust, although the program 
group did report somewhat higher wages and earnings. The program also led to an increase in 
young people’s civic engagement, but had few effects on other youth-development outcome 
measures or on involvement in the criminal justice system. 

The effects on education do suggest that the program may lead to longer-term effects on 
work and earnings. The next report will assess whether the increases in GED receipt and 
postsecondary enrollment are maintained 48 months after study enrollment. It is possible that 
these effects could increase in size, and that boosted postsecondary enrollment could translate 
into earning more degrees and improved job prospects. 

The effects found here fall within the range of effects found for other programs. They 
are similar to or more positive than findings from other nonresidential programs, but somewhat 
smaller than the effects found for Job Corps and National Guard Youth ChalleNGe after a 
similar amount of time. ChalleNGe, for example, led to much larger increases in GED receipt 
and college enrollment, although that program served a younger population that had probably 
been engaged in school more recently. Job Corps, on the other hand, did not increase college 
enrollment but led to somewhat larger increases in GED receipt and a much larger increase in 
vocational certificate receipt. However, both of these programs are residential: Nearly all young 
people in those programs lived at the centers while they were participating. Residential pro- 
grams have a number of advantages. They are more intensive, they pull young people away 
from sometimes disruptive home and community environments, and they can take advantage of 
positive group dynamics. But they are also not an option for young people who are not willing 
or able to live away from home, such as those with children (like 30 percent of the young 
people in the YouthBuild study when they enrolled). Residential programs are also more 
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expensive, so YouthBuild is likely to cost less than these other two programs. (A fomial cost 
analysis of YouthBuild will be presented in the final impact report.) 

The goal of the evaluation is to assess YouthBuild’s effects on the young people it 
serves. In interpreting the program’s effects, it is important to keep in mind the group of young 
people the program served and the actual comparison being made between the YouthBuild 
program group and the control group. YouthBuild served a group of young people who were 
disadvantaged, but highly motivated. The program’s screening processes are designed to ensure 
that young people who enter the program have a good chance of completing it. As a result, 
young people who made it through the screening process and into the study (in both the pro- 
gram and control groups) were probably more motivated and persistent at the time they applied 
than the typical young person who has not completed high school. This level of motivation can 
be seen in the control group’s high rate of participation in services during the follow-up period. 
By 30 months, for example, 72 percent of the control group had participated in education. Thus, 
this evaluation is estimating the effects of YouthBuild by comparing YouthBuild participants 
with similarly motivated young people who sought out other services in their communities. It is 
not estimating YouthBuild’s effects relative to a group who received no services. This compari- 
son is still a fair test, since the hypothesis is that YouthBuild should lead to better outcomes 
because it provides all of these services in one model and in the context of a strong focus on 
youth empowerment and development. 

The evaluation is also a fair test in the sense that most programs in the study were found 
to have implemented the program with high overall fidelity, as documented in the earlier report. 
In other words, the effects presented here are a good assessment of YouthBuild as it was 
intended to operate. Nonetheless, that earlier report found that certain program components, 
particularly postsecondary preparation and youth-leadership services, were implemented less 
consistently than the other components, such as education and vocational training. In addition, 
even programs that achieved high fidelity in a certain program component often implemented it 
in different ways, adapting to their local circumstances. The final report will examine how 
certain variations in program features might affect YouthBuild’s impacts. 

Although it is too early to make judgments about YouthBuild’s effects overall or the ef- 
fects of certain types of programs, the findings here suggest that there may be room for im- 
provement in at least a few areas: cultivating and identifying job openings for participants, 
preparing them for college, and maintaining contact with them after they leave the program. 
These components were rated the lowest by program participants. The lack of strong effects on 
employment suggests that the program could do a better job of helping participants move into 
work. More broadly, the services provided to young people after they leave the program could 
be strengthened or broadened. More efforts could be made to engage young people soon after 
they leave the program, and to keep them in in contact with staff members. 
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In some cases, programs are already beginning to make changes that are likely to 
strengthen their impact, for example by putting increasing emphasis on creating connections 
with colleges. In an era when higher education or training is required to land a decent-paying 
job, this shift in emphasis seems like a move in the right direction. Similarly, many programs 
are beginning to offer vocational training not only in construction but in other areas as well. 
While construction training is integral to YouthBuild’s identity and helps it to achieve its 
community service goal by building housing for low-income communities, it is possible that 
exclusively focusing on construction could limit YouthBuild’s effects. The recent downturn in 
the housing market made it very difficult for many programs to provide adequate training 
opportunities for young people and also probably limited programs’ ability to place young 
people in related jobs. Very few (about 10 percent) of the young people in the YouthBuild 
group who were working at 30 months reported working in construction. In recent years, many 
programs have started offering training in other areas in addition to construction, such as in 
infonnation technology or health care. As noted earlier, in 2014, the U.S. Department of Labor 
(DOL) began allowing selected grantees (those who had received prior grants) the option of 
offering a “construction plus” model, in which training is offered for construction but also for 
other high-demand jobs. 

For some programs, one key to improving services is related to funding. In general, the 
implementation report documented that funding stability was an important issue for programs. 
Most programs did not have funding to sustain operations for several years. Instead, they were 
heavily reliant on receiving DOL grants every few years that require them to match 25 percent 
of the grant with nonfederal resources. As noted in Chapter 2, 13 programs in the evaluation 
shut down after serving the cohort of young people enrolled in the study. Unstable funding 
inevitably affects staff stability. Inconsistent or uncertain program funding means that staff 
positions may have to be eliminated or that staff members may seek other opportunities. As 
discussed in the earlier implementation report, many programs said that they did not have the 
staff time to devote to cultivating and identifying jobs for participants or to other transition 
services, and those with larger budgets generally did better in these areas. Similarly, staff 
turnover affected young people’s engagement in follow-up services, since new staff members 
would not have relationships with program graduates built while they were attending the 
program. 

Finally, another important part of the context for the evaluation is the economy. Al- 
though the Great Recession had officially ended by the time these young people entered the 
study and participated in YouthBuild, employment and wages had not yet recovered. Unem- 
ployment rates remained high for young people during the follow-up period for this report, and 
they were especially high for young people without high school diplomas and for black and 
Latino young people. One of YouthBuild’s goals is to help these young people get an early 
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advantage in the labor market. A later report from the evaluation will assess whether the 
increases in education and training observed so far will have longer-term effects. 
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Appendix A 


Site Selection, Random Assignment, 
the Analysis Model, and Previous Evaluations 



THIS PAGE INTENTIONALLY LEFT BLANK 



This appendix describes the processes used to select YouthBuild programs to participate in the 
evaluation, and describes how random assigmnent procedures were implemented at each 
program. It also includes a discussion of the effect of random assigmnent on recruitment, 
eligibility, and enrollment. The final section discusses the impact analysis model’s specifica- 
tions, including weighting and the handling of missing data. 


Site Selection 

Not all programs receiving U.S. Department of Labor (DOL) or Corporation for National and 
Community Service (CNCS) funding at the start of the evaluation could be included in the 
evaluation, either because they were unable to continue providing services during the period in 
which study participants were to be enrolled, or because of other concerns about their suitabil- 
ity. Thus the first step in the evaluation was to select programs for inclusion. Deciding on the 
total number of programs to include in the impact component of the evaluation required a 
balance of three objectives: (1) maximizing the representativeness of the sample and the 
statistical power of the impact analysis, (2) ensuring high-quality implementation of program 
enrollment and random assigmnent procedures, and (3) evaluation budget considerations. 
Ultimately, 75 programs were included in the evaluation. Fifty-eight of these were selected 
from the programs awarded grants by DOL in 2011, and 17 were selected from programs that 
did not receive DOL funding in 201 1 but did receive funding from CNCS. 1 The latter programs 
are referred to here as CNCS-funded programs, although they might have received funding 
from other, non-DOL sources. 2 

As shown in Appendix Table A.l, the programs participating in the study look very 
similar to all programs funded by DOL and CNCS in 201 1 . 

Selecting DOL-Funded Programs 

DOL awarded grants to 74 YouthBuild programs in May 201 1. 3 Of these 74, 3 pro- 
grams were deemed to be a poor fit for the evaluation because young people assigned to the 
control group were likely to receive substantially the same services as those in the program 
group. Among these programs were ones that operated in conjunction with the Conservation 
Corps, and ones embedded in charter schools where control group members could remain in the 


DOL and CNCS chose to include the CNCS-funded programs in the evaluation in order to examine 
whether DOL-funded programs have different impacts than CNCS-funded programs. 

‘A number of these programs subsequently received funding from DOL as part of the 2012 funding cycle. 

3 An additional two programs received funding to supplement their March 2011 grants. These two pro- 
grams were not considered part of the May grantee class. 
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Appendix Table A.l 


Representativeness of Study Programs 


Program Response 

Percentage of 
Study Programs 

Percentage of 
All Programs 3 

Years operating YouthBuild 

1-3 

0.0 

4.5 

4-6 

32.0 

33.6 

7-10 

25.3 

22.7 

More than 10 

42.7 

39.1 

Yearly operating budget b 

Missing 

8.0 

9.1 

Less than $500,000 

29.3 

30.0 

Between $500,000 and $1 million 

38.7 

38.2 

$1 million and above 

24.0 

22.7 

Funders c 

DOL 

93.3 

90.0 

CNCS 

45.3 

52.7 

Private foundation 

52.0 

47.3 

State or locality 

57.3 

54.5 

Some other entity 

14.7 

13.6 

Predominant funder 

DOL 

0.0 

0.0 

CNCS 

0.0 

0.0 

Private foundation 

1.3 

0.9 

State or locality 

5.3 

6.4 

Some other entity 

0.0 

0.0 

Number of years program director has worked at YouthBuild 

Less than 1 

8.0 

9.1 

1-3 

25.3 

26.4 

4-6 

26.7 

27.3 

7-10 

16.0 

15.5 

More than 10 

24.0 

21.8 

Sample size 

75 

110 


(continued) 
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Appendix Table A.l 


SOURCE: Calculations based on the YouthBuild grantee survey. 

NOTES: Due to rounding, the percentages in some categories may not sum to 100 percent. 

“All programs that received funding from the DOL or CNCS in 201 1. 

b This amount represents the YouthBuild program's operating budget for the fiscal year when the 2011 
YouthBuild grants were awarded. The awards were announced May 201 1 and were received in the months 
that followed. 

“Categories are not mutually exclusive, as some programs reported having multiple funders. 


school and also receive some type of vocational training. Including these programs in the 
evaluation would not have provided a true test of YouthBuild’s effects, since the program and 
control groups would have received nearly identical services. The final sample frame for 
selection of DOL-funded programs thus included 7 1 programs. 4 

Given budget constraints, 60 of these programs were selected to participate in the eval- 
uation using probability-proportional-to-size sampling. Each program had a probability of 
selection that was proportional to its expected enrollment in a given program year. This method 
gave each YouthBuild slot (or young person served) an equal chance of being selected for the 
evaluation, meaning that the resulting sample of young people who enrolled in the study should 
be representative of the young people served by these programs. All of the 60 selected programs 
were required by DOL to participate in the evaluation. Of these, however, the study team 
determined during initial discussions with program staff members that 2 programs would be 
unable to enroll any study-group participants during the intake period. The final sample of DOL 
programs was thus 58. 

Selecting CNCS-Funded Programs 

CNCS funds programs through its National Direct grant to YouthBuild USA. Forty 
YouthBuild programs received CNCS grants but not DOL funding in 2011. After reviewing the 
available information and conducting phone calls with each of the 40 programs, the evaluation 
team determined that many of these programs, particularly those receiving small CNCS grants, 
were likely to shut down in 2012 or not enroll young people during the study enrollment period. 
For this reason the study team, along with DOL’s Employment and Training Administration 


4 According to their grant proposals, the 3 excluded programs planned to serve a total of 133 young people 
in a given program year. The other 71 programs planned to serve a total of 3,171 in a given program year. 
Since the excluded programs accounted for only 4.1 percent of the expected enrollment among DOL-funded 
programs, the study team’s ability to extrapolate the study findings to all DOL-funded programs is not 
compromised. 
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and CNCS staff members, opted to select the 24 programs that received CNCS grants of at least 
$95,000 in 2010. Of these 24, 4 programs subsequently determined that they would shut down 
or otherwise be unable to enroll new participants during the intake period. An additional 3 
programs were deemed to be unsuitable for the evaluation because they operated in areas where 
control group members would be very likely to receive services similar or identical to those 
received by the program group. The resulting sample of CNCS programs was thus 17. 


Developing and Implementing Random Assignment Procedures 

Once YouthBuild programs were selected for participation in the random assignment study, the 
study team visited each of them to meet with its leaders and program staff members to further 
explain the study, answer questions, and begin developing plans for the random assignment of 
young people. 

The study team was flexible about when random assigmnent was conducted, relative to 
programs’ recruitment activities. Random assigmnent could be conducted before, during, or 
after Mental Toughness Orientation. Decisions about the timing of random assigmnent were 
made in partnership with the program, with the goal of conducting random assignment after the 
point in the recruitment process when a program experienced the largest drop-off, so as to 
maximize the possibility that young people in the program group would ultimately emoll in 
YouthBuild. It was also important, however, to ensure that random assignment was not placed 
so late in the process that the control group would have experienced a significant portion of the 
program. For example, the team avoided placing random assigmnent toward the end of a 
lengthy Mental Toughness Orientation. 

Once the study team and a program developed a random assigmnent plan together, the 
study team customized a research procedures manual for that program’s staff. This manual 
detailed the research design and the steps required of program staff members at each step from 
outreach through enrollment. Members of the study team usually conducted another site visit to 
train all staff members in these procedures, including the procedures for entering data into the 
MDRC random assignment system; these data included basic identifying information about 
study participants such as their names and Social Security numbers. 5 


s Each local program was given a number of “wild cards” that it could use to allow certain applicants it 
selected to bypass random assignment and be allowed to participate. This option was used, for example, when 
a young person’s situation was particularly compelling or when a family member was already a YouthBuild 
participant. Each program was allowed to use 5 percent of its program slots for wild cards. The minimum each 
program received was one wild card. 
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The study team was in communication with programs regularly to monitor their pro- 
gress toward their outreach and recruitment goals, and to monitor the drop-off from application 
to enrollment. If a program was having challenges with recruitment, the study team worked 
with that program to brainstorm ways to improve its numbers. For example, a program might do 
more outreach, delay the start of certain processes (like Mental Toughness Orientation), or 
engage in multiple rounds of recruitment. The study team offered advice and support through- 
out; YouthBuild USA coaches and others were also helpful advisers when programs were 
experiencing challenges. 

Seventy-two programs successfully completed random assignment at least once during 
the evaluation enrollment period of August 201 1 to January 2013. The study team allowed 37 
programs that had difficulty reaching their recruitment targets to enroll young people without 
going through random assignment for at least one enrollment cycle, sometimes several. Pro- 
grams might request to bypass random assignment when not enough applicants were present on 
the day of random assignment, when they felt they needed to focus on meeting their DOL or 
CNCS grant recruitment benchmarks, or when they were experiencing significant delays in 
starting their program cycles because they could not recruit enough young people. Three 
programs were never able to conduct random assignment due to low recruitment numbers. 

Appendix Table A.2 presents the baseline characteristics for the full sample and then 
the program and control groups created through the random assignment process. As would be 
expected with random assignment, there are few differences between the two groups with 
respect to these baseline characteristics. 


The Analysis Model 

The basic estimation strategy is to compare average outcomes for the program and control 
groups. Regression adjustment in a linear regression model increases the power of the statistical 
tests. 


Outcome data were processed according to standard procedures to check for outliers or 
other irregularities. Outlier values on employment-related outcome variables (such as earnings, 
hourly wages, and weekly hours worked) were set to missing. These outlier values affected less 
than 1 percent of the sample for any given outcome. 

The impact analysis used the following model: 

Yij = a + pPy + SXjj + yj + sy, 

Where Yy is the outcome of interest (such as “earned a high school equivalency creden- 
tial,” “employed,” or “involved in civic activities”) for sample member i in site j, 
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a is the intercept of the regression, 

Pjj is an indicator for membership in the program group, 

Xjj is the series of binary variables that represents the baseline covariates for a sample 
member, including age, gender, whether the person applied to YouthBuild at a CNCS-funded 
program, highest grade completed, race/ethnicity, whether or not a young person was a parent, 
and high school diploma or equivalency completion, for sample member i in site j, 

8 is the set of regression coefficients for X y , 

y represents program fixed effects to account for varying random assignment ratios by 

site, 


and Sy is the random error term for sample member i in site j. 

For the analysis of survey outcomes, weights were added to the model to account for 
varying selection probabilities by cohort and research group. 

For an observation with a missing baseline covariate (see the list for Xy above), that co- 
variate was assigned the average sample value and a dummy variable indicating “missing” for 
that covariate was set to 1 and thus included in the analysis model with Xy. Fewer than 2 
percent of observations had missing values for any given covariate except parent status. Parent 
status was missing for 1 1 percent of the sample. Observations with missing values for an 
outcome variable were dropped from the impact analysis for that outcome. Missing values for 
outcome variables were not imputed. 


Previous Evaluations 

Appendix Table A.3 summarizes some previous evaluations of programs for young people 
without high school diplomas. 
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Appendix Table A.2 


Baseline Characteristics by Research Group 


Characteristic (%) 

Full 

Sample 

YouthBuild 

Group 

Control 

Group 

Age 



* 

16-18 years old 

33.0 

34.2 

30.3 

19-21 years old 

46.3 

45.5 

48.0 

22 years old or older 

20.7 

20.3 

21.6 

Male 

64.1 

63.9 

64.7 

Race/ethnicitv a 



* 

Hispanic or Latino 

14.6 

14.6 

14.4 

White, non-Hispanic 

15.3 

15.6 

14.6 

Black, non-Hispanic 

62.9 

62.9 

63.1 

Other b 

6.0 

6.0 

5.9 

Not specified 

1.1 

0.8 

1.8 

Has a child 

30.0 

29.0 

32.1 ** 

Highest grade completed 0 




6th or lower 

0.4 

0.4 

0.4 

7th 

1.0 

1.0 

0.9 

8th 

7.5 

7.7 

6.8 

9th 

18.6 

18.9 

18.0 

10th 

26.2 

26.4 

25.6 

11th 

34.9 

34.2 

36.5 

12th 

10.0 

9.8 

10.4 

Has a high school diploma or equivalent 

9.2 

8.8 

10.0 

Has a diagnosed disability (learning or physical) 

10.6 

11.1 

9.5 

Housing status 




Lives with family 

61.0 

61.7 

59.5 

Owns/rents apartment, room, or house 

15.2 

15.2 

15.2 

Is staying at someone's apartment, room, or house 

15.7 

15.2 

16.8 

Is staying with foster guardian/in foster system 

0.6 

0.7 

0.4 

Lives in a halfway house/transitional house 

1.2 

1.0 

1.5 

Is in residential treatment 

0.3 

0.3 

0.2 

Is homeless 

3.0 

2.7 

3.5 


(continued) 
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Appendix Table A.2 (continued) 


Characteristic (%) 

Full 

Sample 

YouthBuild 

Group 

Control 

Group 

Who suggested you apply to YouthBuild? 

Family member or relative 

29.8 

29.0 

31.3 

No one 

32.5 

32.9 

31.6 

School counselor, truant officer, teacher, or principal 

4.3 

4.3 

4.1 

Friend 

20.7 

20.6 

20.7 

Other 

9.7 

9.9 

9.2 

Reasons for applying to YouthBuild 

GED credential 

87.7 

88.0 

87.0 

College 

63.1 

62.6 

64.4 

To get life on track 

88.2 

88.5 

87.6 

Job 

84.6 

85.1 

83.5 

Training 

67.2 

66.7 

68.2 

Friends 

7.0 

7.4 

6.2 

Because of children or the need to support family 

1.5 

1.5 

1.6 

Other 

4.4 

4.6 

4.0 

Sample size 

3,929 

2,700 

1,229 


SOURCE: Calculations based on the YouthBuild baseline data form. 

NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 

* = 10 percent. 

GED = General Educational Development. 

“Categories are mutually exclusive. 

b "Other" includes Hawaiian Native or other Pacific Islander, Asian, American Indian or Alaskan, and 
responses of multiple races/ethnicities. 

L This information is missing for some sample members. 
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Appendix Table A.3 

Selected Evaluations of Programs for Young People Without High School Diplomas 


Evaluation (Dates) 

Target Group 

Program Model 

Sample Size (Number of 

Sites) 

Summary of Results 

Nonresidential 

American Conserva- 
tion and Youth 

Service Corps 
(1993-1996) 

Mostly 18- to 25-year- 
old out-of-school 
young people 

Paid work experience in 
community service projects, 
education and training, support 

services 

1,009 young people (4 sites) 

Increases in employment and 
decreases in arrests, particularly for 
African-American men 

Conservation Corps 
Evaluation (2011) 

Mostly 18- to 25-year- 
old out-of-school 
young people, mostly 
students of color 

Temporary, full-time, subsi- 
dized work in community 
service projects; basic adult 
education; opportunities to earn 
college credit; case manage- 
ment; job-readiness skills 

2,043 young people 
(21 sites) 

No significant impacts on 
employment or being in school 

JOB START 
(1985-1993) 

17- to 21 -year-old high 
school dropouts with 
low reading levels 

Education, training, support 
services, job-placement 
assistance 

1,914 young people 
(13 sites) 

Increases in GED receipt; few 
impacts on labor-market outcomes 
(except at one site) 

National Job Training 
Partnership Act (out- 
of-school young 
people analysis) 
(1987-1994) 

Disadvantaged 16- to 

21 -year-old out-of- 
school young people 

Education, job-skills training, 
job placement, on-the-job 
training, and support services 

5,690 young people 
(16 sites) 

No earnings impacts for women or 
for men who had not been arrested; 
possibly negative impacts for men 
who had been arrested 

Center for 

Employment Training 

Replication 

(1995-1999) 

Disadvantaged, out-of- 
school young people 
ages 16 to 21 

Education and vocational 
training 

1,485 young people 
(12 sites) 

Few impacts on employment and 
earnings overall; some impacts for 
younger participants 


(continued) 




Appendix Table A.3 (continued) 


Evaluation (Dates) 

Target Group 

Program Model 

Sample Size (Number of 

Sites) 

Summary of Results 

Residential 

Job Corps 
(1994-2003) 

Disadvantaged young 
people ages 16 to 24 

Employment, education, and 
training in a (mostly) 
residential setting 

15,386 young people 
(nationwide) 

Earnings and employment impacts in 

Y ears 3 and 4 of the study period; 
impacts faded after Y ear 4, according 
to administrative data; results appear 
stronger for older participants (those 

20 to 24 years old) 

National Guard Youth 

ChalleNGe 

(2005-2011) 

High school dropouts 
ages 16 to 18 who are 
drug free and not 
heavily involved with 
the justice system 

Education, service to commu- 
nity, and other components in a 
quasi-military residential 
setting; 12-month 
postresidential mentoring 
program 

1,173 young people 
(10 sites nationwide) 

Impacts on GED receipt, 
postsecondary enrollment, and 
employment and earnings at the 
three-year follow-up point 


SOURCES: Maynard (1980); Gueron (1984); Jastrzab, Masker, Blomquist, and Orr (1996); Cave, Bos, Doolittle, and Toussaint (1993); Orr et al. 
(1997); Quint, Bos, and Polit (1997); Miller et al. (2005); Schochet, Burghardt, and McConnell (2008); Millenky, Bloom, Muller-Ravett, and 
Broadus (2011); Price et al. (2011). 
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The YouthBuild 12- and 30- month surveys provide information about the YouthBuild sample 
members on topics such as participation in training and education, employment and job charac- 
teristics, youth development, and other outcome measures. As the survey was administered to a 
subset of the YouthBuild sample, it is necessary to assess the reliability of impact results for the 
survey sample in two ways. First, the results for the survey sample may or may not generalize to 
(or be representative of) the full sample because (1) only a subset of the YouthBuild sample was 
selected to be interviewed and (2) individuals who responded to the surveys may be different 
from those who were selected for the survey but did not respond. Second, the failure of some 
sample members to respond to the surveys may compromise the validity of the impact esti- 
mates, particularly if the program and control groups responded to the survey at different rates. 

This appendix presents a description of the survey fielding efforts, assesses whether the 
survey can be generalized to the research sample, and assesses its validity for estimating 
program impacts. Overall, the results suggest that the survey samples provide valid estimates of 
the program’s effects that can be generalized to the research sample. 


Sample Selection and Survey Administration 

The research sample includes 3,929 sample members. Due to budget constraints, only 3,436 
people in the full sample could be selected to be interviewed for the survey (that is, to be in the 
fielded sample), as described in Appendix Box B.l. This fielded sample is used for all surveys 
in this evaluation. 

All research sample members who were randomly assigned between August 20 1 1 and 
February 2012 were included in the fielded sample. It was necessary to include all research 
sample members enrolled during those months in the fielded sample because the 12-month 
survey began before the total research sample size was known. When study enrollment ended in 
February 2013, the total research sample was large enough that it was necessary to select a 
subsample of those randomly assigned from March 2012 through January 2013. Specifically, all 
control group members were included and 76 percent of program group members were random- 
ly selected from each program. This sampling plan ensured that each program was represented 
in the survey analysis and helped achieve a more balanced sample of program and control group 
members. 

Appendix Table B.l shows baseline characteristics for the research sample (3,929), the 
fielded sample (3,436), and the nonfielded sample (493). Overall, there are very few statistically 
significant differences between the fielded and nonfielded samples. Fielded sample members 
were more likely than nonfielded sample members to have had children when they enrolled in 
the study, but less likely to have had a diagnosed disability. 
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Appendix Box B.1 

Sample Definitions 

Research sample: All 3,929 sample members who were randomly assigned during the 
sample intake period, which extended from August 2011 through January 2013. 

Fielded sample: The total of 3,436 research sample members who were selected for the 
surveys. 

Nonfielded sample: The 493 research sample members who were not selected for the 
surveys. 

Respondent sample: Fielded sample members who completed a given follow-up survey. 

Nonrespondent sample: Fielded sample members who did not complete a given follow- 
up survey for various reasons, for example, because they could not be located or because 
they refused to be interviewed/ 


The nonrespondent sample at 12 months includes 9 deceased sample members, 63 incarcerated 
sample members, and 3 sample members in active military service. At 30 months, the nonrespondent 
sample includes 15 deceased sample members, 76 incarcerated sample members, and 2 sample members 
in active military service. 


The 12-month survey was fielded (administered to survey recipients) between Decem- 
ber 2012 and May 2014. The 30-month survey was fielded between January 2014 and Decem- 
ber 2015. In both, sample members were asked to complete the survey online; sample members 
who did not do so were then called on the phone and asked to complete the survey that way. If a 
sample member still could not be reached, a field representative of the survey firm followed up 
in person. It took slightly different amounts of time to complete the survey online, by phone, or 
in person, but on average sample members completed the survey in less than 43 minutes. 


Characteristics of Respondents and Nonrespondents 
in the Fielded Sample 

Of the 3,436 young people who were chosen to be surveyed, 2,784 completed the 12-month 
follow-up survey and 2,756 completed the 30-month follow-up survey, response rates of 81 
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Appendix Table B.l 


Selected Baseline Characteristics of the Fielded and Nonfielded Samples 


Characteristic (%) 

Full 

Sample 

Fielded 

Sample 

Nonfielded 

Sample 

Age 

16-18 years old 

33.0 

32.9 

33.7 

19-21 years old 

46.3 

46.0 

48.1 

22 years old or older 

20.7 

21.0 

18.3 

Male 

64.3 

64.2 

64.7 

Race/ethnicitv a 

Hispanic or Latino 

14.6 

14.3 

16.5 

White, non-Hispanic 

15.3 

15.4 

14.8 

Black, non-Hispanic 

63.0 

63.0 

63.0 

Other b 

6.0 

6.0 

5.5 

Not specified 

1.1 

1.2 

0.2 

Has a child 

33.7 

34.2 

29.6* 

Highest grade completed 0 

6th or below 

0.4 

0.4 

0.6 

7th 

1.0 

1.0 

1.2 

8th 

7.6 

7.9 

5.6 

9th 

18.9 

19.2 

16.7 

10th 

26.6 

26.2 

29.1 

11th 

35.5 

35.5 

35.1 

12th 

10.1 

9.9 

11.8 

Has a high school diploma or equivalent 

9.3 

9.0 

10.9 

Has a diagnosed disability (learning or physical) 

11.1 

10.7 

13.8* 

Housing status 

Lives with family 

63.0 

62.8 

64.1 

Owns/rents apartment, room, or house 

15.7 

15.7 

15.6 

Is staying at someone's apartment, room, or house 

16.2 

16.3 

15.4 

Is staying with foster guardian/in foster system 

0.6 

0.6 

0.6 

Lives in a halfway house/transitional house 

1.2 

1.2 

1.1 

Is in residential treatment 

0.3 

0.2 

0.4 

Is homeless 

3.0 

3.1 

2.7 

Other 

0.0 

0.0 

0.0 


(continued) 
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Appendix Table B.l (continued) 


Characteristic (%) 

Full 

Sample 

Fielded 

Sample 

Nonfielded 

Sample 

Who suggested applying to YouthBuild 

Family member or relative 

30.7 

30.7 

30.7 

No one 

33.6 

34.1 

30.1 

School counselor, truant officer, teacher, or principal 

4.4 

4.4 

4.6 

Friend 

21.3 

21.1 

23.0 

Judge or someone from the justice system 

3.3 

3.3 

3.5 

Someone else 

4.3 

4.3 

4.2 

Case manager, counselor, mentor, or program staff member 

2.4 

2.2 

4.0 

Reasons for applying to YouthBuild 

GED credential 

90.9 

91.3 

88.8 

College 

66.0 

66.1 

65.0 

To get life on track 

92.1 

92.2 

91.2 

Job 

88.7 

89.0 

86.9 

Training 

71.4 

71.6 

70.4 

Friends 

7.7 

7.8 

7.2 

Sample size 

3,929 

3,436 

493 


SOURCE: Calculations based on the YouthBuild baseline data form. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
GED = General Educational Development. 
a Categories are mutually exclusive. 

b "Other" includes Hawaiian Native or other Pacific Islander, Asian, American Indian or Alaskan, and 
responses of multiple races/ethnicities. 

L This information is missing for some sample members. 
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percent and 80 percent, respectively; 2,511 completed both the 12- and 30-month surveys. 1 For 
both surveys, the program group had slightly higher response rates than the control group (for 
example, for the 30-month survey the response rate was 81 percent for the program group and 
78 percent for the control group). The differential was not greater than 3 percentage points and 
the differences between response rates were not statistically significant in either survey. 

Appendix Tables B.2 and B.3 present selected baseline characteristics of survey re- 
spondents and nonrespondents for the two surveys. Some differences are to be expected, since 
individuals who respond to surveys tend to be different from those who do not. Nonrespondents 
are often people in harder-to-reach groups, such as those with lower incomes and greater 
mobility. The table illustrates these types of differences. In both surveys, for example, the 
respondent sample has a higher percentage of women than the nonrespondent sample. This 
finding is not surprising, as women often respond to surveys at higher rates than men. 2 The 
respondent and nonrespondent samples for both surveys also had different racial and ethnic 
makeups and different housing statuses. Respondents to the 30-month survey were less likely 
than nonrespondents to have reported a disability when they enrolled in the study. 

These differences were also tested in a logistic model, in which the probability of re- 
sponse was regressed on the baseline covariates shown in Table B.2. A test of joint significance 
is statistically significant for each survey. The differences between the respondent and nonre- 
spondent samples suggest that some caution should be exercised when generalizing the survey 
findings to the research sample. However, because the response rate was fairly high (nonre- 
spondents represent less than 20 percent of the fielded sample), the respondent sample still 
looks similar to the fielded sample. 


Comparisons Between the Research Groups in the Survey 
Respondent Sample 

Although random assignment research designs minimize potential bias, there is a possibility that 
the selective nature of the survey-response process could result in differences between the 
characteristics of the program group and the control group. If such differences arise, they could 
make the impact estimates derived from the respondent sample less reliable. 


'The main text of the report and the impact analysis use a broader measure of survey response than the 
analysis presented in this appendix. (For example, 2,845 sample members answered at least some questions on 
the 12-month follow-up survey and 2,808 sample members did so for the 30-month follow-up survey.) This 
appendix and supporting analyses use survey completion to define “respondents.” Results using the broader 
measure are similar to those shown here. 

2 Groves (2006). 
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Appendix Table B.2 


Selected Baseline Characteristics of Respondents and 
Nonrespondents to the 12-Month Survey 


Characteristic (%) 

Respondents 

Nonrespondents Full Sample 

Age 




16-18 years old 

32.8 

33.3 

33.0 

19-21 years old 

46.4 

44.3 

46.3 

22 years old or older 

20.7 

22.4 

20.7 

Male 

62.2 

72.5 

64.3*** 

Race/ethnicitv a 



*** 

Hispanic or Latino 

13.9 

16.0 

14.6 

White, non-Hispanic 

15.5 

15.0 

15.3 

Black, non-Hispanic 

64.1 

58.6 

63.0 

Other b 

5.4 

8.6 

6.0 

Not specified 

1.1 

1.8 

1.1 

Has a child 

33.3 

38.5 

33.7** 

Highest grade completed 0 




6th or below 

0.3 

0.6 

0.4 

7th 

1.0 

0.8 

1.0 

8th 

8.2 

6.6 

7.6 

9th 

19.3 

18.4 

18.9 

10th 

25.6 

28.8 

26.6 

11th 

35.3 

36.4 

35.5 

12th 

10.2 

8.4 

10.1 

Has a high school diploma or equivalent 

9.4 

7.5 

9.3 

Has a diagnosed disability (learning or physical) 

10.6 

11.5 

11.1 

Housing status 



*** 

Lives with family 

64.1 

57.4 

63.0 

Owns/rents apartment, room, or house 

15.8 

15.3 

15.7 

Is staying at someone's apartment, room, or house 

15.0 

21.7 

16.2 

Is staying with foster guardian/in foster system 

0.6 

0.6 

0.6 

Lives in a halfway house/transitional house 

1.1 

1.6 

1.2 

Is in residential treatment 

0.1 

1.0 

0.3 

Is homeless 

3.3 

2.2 

3.0 


(continued) 
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Appendix Table B.2 (continued) 


Characteristic (%) 

Respondents 

Nonrespondents Full Sample 

Who suggested applying to YouthBuild 

Family member or relative 

31.3 

28.4 

30.7 

No one 

34.6 

31.9 

33.6 

School counselor, truant officer, teacher, or principal 

4.1 

5.6 

4.4 

Friend 

20.7 

22.8 

21.3 

Judge or someone from the justice system 

3.1 

4.1 

3.3 

Someone else 

4.1 

5.3 

4.3 

Case manager, counselor, mentor, or program staff member 

2.2 

1.9 

2.4 

Reasons for applying to YouthBuild 

GED 

91.2 

91.4 

90.9 

College 

65.9 

66.9 

66.0 

To get life on track 

92.2 

92.3 

92.1 

Job 

88.9 

89.4 

88.7 

Training 

70.9 

74.5 

71.4* 

Friends 

7.5 

9.1 

7.7 

Sample size 

2,784 

652 

3,929 


SOURCE: Calculations based on the YouthBuild baseline data form. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
“Categories are mutually exclusive. 

b " Other" includes Hawaiian Native or other Pacific Islander, Asian, American Indian or Alaskan, and responses 
of multiple races/ethnicities. 

L This information is missing for some sample members. 
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Appendix Table B.3 

Selected Baseline Characteristics of 
Respondents and Nonrespondents to the 30-Month Survey 


Characteristic (%) 

Respondents 

Nonrespondents Full Sample 

Age 




16-18 years old 

32.7 

34.0 

33.0 

19-21 years old 

46.2 

45.3 

46.3 

22 years old or older 

21.1 

20.7 

20.7 

Male 

61.8 

74.0 

64.3*** 

Race/ethnicitv a 



*** 

Hispanic or Latino 

13.7 

16.8 

14.6 

White, non-Hispanic 

14.7 

18.1 

15.3 

Black, non-Hispanic 

64.8 

56.0 

63.0 

Other b 

5.7 

7.5 

6.0 

Not specified 

1.1 

1.6 

1.1 

Has a child 

34.2 

34.2 

33.7 

Highest grade completed 0 




6th or below 

0.3 

0.7 

0.4 

7th 

1.0 

0.7 

1.0 

8th 

7.8 

8.0 

7.6 

9th 

19.4 

18.2 

18.9 

10th 

26.4 

25.6 

26.6 

11th 

35.0 

37.7 

35.5 

12th 

10.1 

8.9 

10.1 

Has a high school diploma or equivalent 

9.2 

8.4 

9.3 

Has a diagnosed disability (learning or physical) 

10.2 

13.1 


Housing status 



*** 

Lives with family 

64.5 

56.0 

63.0 

Owns/rents apartment, room, or house 

15.4 

17.1 

15.7 

Is staying at someone's apartment, room, or house 

15.4 

19.8 

16.2 

Is staying with foster guardian/in foster system 

0.6 

0.6 

0.6 

Lives in a halfway house/transitional house 

1.0 

2.0 

1.2 

Is in residential treatment 

0.1 

0.9 

0.3 

Is homeless 

3.0 

3.6 

3.0 

Other 

0.0 

0.0 

0.0 
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Appendix Table B.3 (continued) 


Characteristic (%) 

Respondents 

Nonrespondents Full Sample 

Who suggested applying to YouthBuild 

Family member or relative 

31.3 

28.4 

30.7 

No one 

34.5 

32.1 

33.6 

School counselor, truant officer, teacher, or principal 

4.2 

5.3 

4.4 

Friend 

20.8 

22.2 

21.3 

Judge or someone from the justice system 

3.0 

4.2 

3.3 

Someone else 

4.1 

5.0 

4.3 

Case manager, counselor, mentor, or program staff member 

2.0 

2.7 

2.4 

Reasons for applying to YouthBuild 

GED 

90.9 

92.6 

90.9 

College 

66.3 

65.4 

66.0 

To get life on track 

92.3 

92.0 

92.1 

Job 

89.0 

88.7 

88.7 

Training 

71.7 

71.2 

71.4 

Friends 

7.6 

8.4 

7.7 

Sample size 

2,756 

680 

3,929 


SOURCE: Calculations based on the YouthBuild baseline data form. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
a Categories are mutually exclusive. 

b " Other" includes Hawaiian Native or other Pacific Islander, Asian, American Indian or Alaskan, and responses 
of multiple races/ethnicities. 

L This information is missing for some sample members. 


It does not appear that these differences did arise. Selected baseline characteristics for 
program and control group survey respondents are shown in Appendix Tables B.4 and B.5 for 
the 12- and 30-month surveys, respectively. Overall, the two groups look nearly identical. The 
only statistically significant difference that emerged was that control group respondents to the 
12-month survey reported more frequently than the program group that they had applied to 
YouthBuild to earn a high school equivalency credential. Comparisons between the program 
and control groups for those that completed both survey waves also found no significant 
differences. 
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Appendix Table B.4 


Selected Baseline Characteristics of Program and Control Group 
Respondents to the 12-Month Survey 


Characteristic (%) 

YouthBuild 

Group 

Control 

Group 

Age 

16-18 years old 

34.0 

30.7 

19-21 years old 

45.3 

48.6 

22 years old or older 

20.7 

20.7 

Male 

61.9 

62.9 

Race/ethnicitv a 

Hispanic or Latino 

13.7 

14.4 

White, non-Hispanic 

15.7 

15.0 

Black, non-Hispanic 

64.0 

64.3 

Other* 5 

5.8 

4.8 

Not specified 

0.8 

1.6 

Has a child 

32.7 

34.3 

Highest grade completed 0 

6th or below 

0.3 

0.3 

7th 

1.1 

0.9 

8th 

8.5 

7.5 

9th 

19.8 

18.5 

10 th 

25.2 

26.4 

11th 

35.4 

35.1 

12 th 

9.7 

11.2 

Has a high school diploma or equivalent 

8.8 

10.6 

Has a diagnosed disability (learning or physical) 

10.8 

10.1 

Housing status 

Lives with family 

64.8 

62.7 

Owns/rents apartment, room, or house 

15.8 

15.7 

Is staying at someone's apartment, room, or house 

14.6 

15.9 

Is staying with foster guardian/in foster system 

0.7 

0.4 

Lives in a halfway house/transitional house 

0.8 

1.7 

Is in residential treatment 

0.1 

0.1 

Is homeless 

3.2 

3.5 


(continued) 
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Appendix Table B.4 (continued) 


Characteristic (%) 

YouthBuild 

Group 

Control 

Group 

Who suggested applying to YouthBuild 

Family member or relative 

30.0 

33.6 

No one 

35.6 

32.5 

School counselor, truant officer, teacher, or principal 

4.0 

4.3 

Friend 

20.7 

20.7 

Judge or someone from the justice system 

3.2 

2.8 

Someone else 

4.0 

4.1 

Case manager, counselor, mentor, or program staff member 

2.3 

2.0 

Reasons for applying to YouthBuild 

GED 

92.0 

89.7** 

College 

65.8 

66.1 

To get life on track 

92.7 

91.2 

Job 

89.2 

88.3 

Training 

70.1 

72.5 

Friends 

7.8 

6.9 

Sample size 

1,815 

969 


SOURCE: Calculations based on the YouthBuild baseline data form. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Categories are mutually exclusive. 

b "Other" includes Hawaiian Native or other Pacific Islander, Asian, American Indian or Alaskan, 
and responses of multiple races/ethnicities. 

c This information is missing for some sample members. 


91 




Appendix Table B.5 


Selected Baseline Characteristics of Program and Control Group 
Respondents to the 30-Month Survey 


Characteristic (%) 

YouthBuild 

Group 

Control 

Group 

Age 

16-18 years old 

33.7 

30.7 

19-21 years old 

45.0 

48.6 

22 years old or older 

21.3 

20.7 

Male 

61.7 

61.8 

Race/ethnicitv a 

Hispanic or Latino 

13.4 

14.2 

White, non-Hispanic 

14.8 

14.5 

Black, non-Hispanic 

64.8 

64.8 

Other 11 

6.0 

5.0 

Not specified 

0.9 

1.6 

Has a child 

33.8 

35.0 

Highest grade completed 0 

6th or below 

0.2 

0.3 

7th 

1.1 

0.8 

8th 

8.2 

7.2 

9th 

19.8 

18.7 

10th 

26.4 

26.4 

11th 

34.4 

36.0 

12 th 

9.9 

10.7 

Has a high school diploma or equivalent 

8.8 

10.1 

Has a diagnosed disability (learning or physical) 

10.5 

9.6 

Housing status 

Lives with family 

64.9 

63.8 

Owns/rents apartment, room, or house 

15.7 

14.7 

Is staying at someone's apartment, room, or house 

14.7 

16.7 

Is staying with foster guardian/in foster system 

0.7 

0.4 

Lives in a halfway house/transitional house 

1.0 

1.2 

Is in residential treatment 

0.1 

0.0 

Is homeless 

2.9 

3.0 

Other 

0.0 

0.1 


(continued) 
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Appendix Table B.5 (continued) 


Characteristic (%) 

YouthBuild 

Group 

Control 

Group 

Who suggested applying to YouthBuild 

Family member or relative 

30.1 

33.4 

No one 

35.6 

32.7 

School counselor, truant officer, teacher, or principal 

4.2 

4.1 

Friend 

20.6 

21.3 

Judge or someone from the justice system 

3.1 

2.9 

Someone else 

4.2 

4.0 

Case manager, counselor, mentor, or program staff member 

2.2 

1.7 

Reasons for applying to YouthBuild 

GED 

91.5 

89.9 

College 

66.1 

66.5 

To get life on track 

92.8 

91.2 

Job 

89.4 

88.4 

Training 

71.3 

72.4 

Friends 

8.2 

6.5 

Sample size 

1,795 

961 


SOURCE: Calculations based on the YouthBuild baseline data form. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Categories are mutually exclusive. 

b "Other" includes Hawaiian Native or other Pacific Islander, Asian, American Indian or Alaskan, and 
responses of multiple races/ethnicities. 

c This information is missing for some sample members. 
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Consistency of Impacts 

A previous section suggested some caution in interpreting the results of the surveys. Specifical- 
ly, as discussed above, there are some differences in baseline characteristics between the sample 
members who responded to the survey and those who did not. This section helps to put the 
survey results in context by comparing the impacts estimated based on administrative records 
for respondents with the impacts for nonrespondents. These comparisons were carried out for 
both surveys. Comparisons using administrative records provide the best estimate of the 
program’s effects because they use the full program group and control group, not a potentially 
nonrandom subset of survey respondents. If the respondent and nonrespondent survey samples 
have similar impacts estimated using administrative data, it would give more credibility to the 
survey analysis. 

Appendix Tables B.6 and B.7 present the results, showing impacts for college outcomes 
using National Student Clearinghouse enrollment records. (It was not possible to conduct a 
similar test using data from the National Database of New Hires data due to federal restrictions 
on researchers’ access to these data.) Overall, the impacts are nearly identical for each sample at 
each survey. A test of joint statistical significance across all educational outcomes further 
confinned that survey respondents and nonrespondents did not differ across these outcomes. 
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Appendix Table B.6 

Impacts on Education Based on Records Data at 12 Months 


Impact (Percentage Points) 

Respondent 

Sample 

Nonrespondent 

Sample 

Attended college since random assignment 

Enrolled in a 4-year institution 

0.88 

0.33 

Enrolled in a 2 -year institution 

7.75 

7.14 

Enrolled in a less-than-2-year institution 

0.05 

0.36 

Public 

8.25 

7.43 

Private 

0.55 

0.05 

Full time 

2.89 

3.54 

Received a degree 

0.89 

0.82 

Certificate 

0.68 

1.00 

Associate's 

0.19 

0.12 

Bachelor's 

0.00 

-0.40* 

Master's 

0.04 

0.00 

Sample size 

2,784 

1,145 


SOURCE: MDRC calculations based on National Student Clearinghouse data. 

NOTE: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 


95 




Appendix Table B.7 

Impacts on Education Based on Records Data at 30 Months 


Impact (Percentage Points) 

Respondent 

Sample 

Nonrespondent 

Sample 

Attended college since random assignment 

Enrolled in a 4-year institution 

0.42 

1.45 

Enrolled in a 2 -year institution 

7.63 

7.45 

Enrolled in a less-than-2-year institution 

0.23 

-0.15 

Public 

8.01 

7.95 

Private 

0.28 

0.79 

Full time 

2.62 

4.23 

Received a degree 

0.45 

1.67 

Certificate 

0.47 

1.40 

Associate's 

0.19 

0.13 

Bachelor's 

-0.11 

0.00 

Master's 

0.00 

0.00 

Sample size 

2,756 

1,173 


SOURCE: MDRC calculations based on National Student Clearinghouse data. 

NOTE: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
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Appendix C 


Survey Responses About YouthBuild Experiences and 
Service Receipt at 30 Months 
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Appendix Table C.l 

YouthBuild Experience, Program Group Only, at 30 Months 


Outcome (%) 

YouthBuild 

Group 

Ever received YouthBuild services 

80.0 

Current status, among those who ever received YouthBuild services 


Currently enrolled 

2.3 

Graduated from the program 

68.1 

No longer enrolled and did not graduate 

29.6 

Reason for not participating in or not completing the program 


Transportation 

26.5 

Incarceration 

10.1 

Program schedule 

17.5 

Another job 

27.0 

Moved 

21.7 

Birth of a child or child care problems 

18.1 

Health issue or family pressure 

17.6 

Conflict with the program, staff members, or other participants 

15.8 

Expulsion or being asked to leave 

12.9 

Another reason 

32.5 

Among those who participated 


Received a stipend from YouthBuild 

86.3 

Feel close to a YouthBuild staff person 

84.0 

Rated program, program component, or staff favorably 3 


Overall YouthBuild experience 

90.6 

Construction or other job training 

91.4 

Counseling 

86.4 

Leadership training 

88.0 

Help finding a job 

74.7 

Help applying to college 

81.3 

Understanding your needs 

81.4 

Helping you solve problems 

79.8 

Helping you learn 

86.7 

Helping after you left YouthBuild 

64.0 

Staying in contact after you graduated 

67.4 

Sample size 

1,811 


SOURCE: MDRC calculations based on responses to the 30-month survey. 

NOTE: indicates a response of "very good" or "good." The other response options were "okay," "poor," 
and "does not apply to me." 
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Appendix Table C.2 


Impacts on Service Receipt at 30 Months 



Program 

Control 

Difference 


Outcome 

Group 

Group 

(Impact) 

P- Value 

Education-related services 





Ever participated (%) 

81.2 

69.8 

II 4 *** 

0.000 

GED preparation 

60.5 

49.6 

| | Q *** 

0.000 

Academic tutoring (not related to GED preparation) 

22.7 

13.0 

g fa *** 

0.000 

High school diploma prep courses 

31.8 

28.3 

3.4* 

0.056 

Standardized Achievement Test preparation 

28.8 

19.7 

g | *** 

0.000 

College-preparation activities 3 

36.6 

19.3 

17 3 *** 

0.000 

Getting help finding financial aid 

37.7 

22.6 

15.1 *** 

0.000 

Other b 

15.2 

13.5 

1.8 

0.233 

Months participated 

6.0 

4.1 

| g *** 

0.000 

Job or training-related services 





Ever participated (%) 

78.2 

57.8 

20.4*** 

0.000 

Job-skills training program 

50.2 

28.7 

2| 5 *** 

0.000 

On-the-job training in construction or 





another field 

58.4 

24.7 

33 7 *** 

0.000 

Job certification program 

37.4 

17.3 

20.1 *** 

0.000 

Job-search assistance 0 

63.2 

44.7 

18.5*** 

0.000 

Help applying to a vocational training program 11 

40.4 

22.3 

lg | *** 

0.000 

Months participated 

6.2 

3.1 

3 | *** 

0.000 

Personal-development services 





Ever participated (%) 

65.6 

44.4 

2 | | *** 

0.000 

Help or advice from a mentor 

44.0 

26.1 

17 g *** 

0.000 

Life-skills training 6 

38.0 

17.1 

21 . 0 *** 

0.000 

Communication or public-speaking training 

34.4 

11.6 

22 . 8 *** 

0.000 

Leadership-development training 

43.4 

14.9 

28.4*** 

0.000 

Health services 

32.1 

14.9 

17 2 *** 

0.000 

Mental health services 

23.0 

11.7 

H3 *** 

0.000 

Working with a case manager 

35.8 

20.0 

15.8*** 

0.000 

Months participated 

5.6 

3.3 

2 3 *** 

0.000 

Sample size (total = 2,808) 

1,830 

978 



(continued) 
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Appendix Table C.2 (continued) 

SOURCE: MDRC calculations based on responses to the 30-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

“includes college -awareness or college-guidance activities, college -preparation or -transition programs, or 
preparation for college entrance exams. 

'includes attending adult education classes, various certification courses, and college attendance. 
c Includes activities such as help filling out an application, writing a resume, and going for an interview, 
"'includes help with a program application or interview. 

“Includes activities such as parenting-skills classes or learning how to balance a checkbook. 
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Appendix D 


Survey-Based Impacts and Subgroup Impacts at 
12 Months and Selected Impacts Per Participant 
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Appendix Table D.l 


Impacts on Education and Training at 12 Months 



YouthBuild Control 

Difference 


Outcome (%) 

Group 

Group 

(Impact) 

P-Value 

Earned a high school diploma or GED credential 

37.8 

23.3 


0.000 

High school diploma 

11.7 

12.1 

-0.4 

0.718 

GED 

26.1 

11.3 


0.000 

Enrolled in high school or GED 
classes since random assignment 

64.3 

54.9 

9 4 *** 

0.000 

Enrolled in vocational school since random assignment 3 

25.9 

14.9 

1 1 o*** 

0.000 

Received trade license/training certificate 13 

3.3 

1.0 

2 2 *** 

0.002 

Enrolled in postsecondary courses since random assignment 

15.9 

11.1 

4 g*** 

0.001 

2 -year or community college 

14.3 

10.6 


0.007 

4-year college or university 

2.9 

1.1 

j g*** 

0.004 

Received a postsecondary degree 

0.6 

0.2 

0.3 

0.223 

Associate's degree 

0.4 

0.3 

0.2 

0.548 

Bachelor's degree 

0.1 

0.1 

- 0.1 

0.562 

Other degree 

0.2 

0.1 

0.1 

0.454 

Currently enrolled in school 

24.4 

20.9 

3.4** 

0.043 

GED prep or high school classes 

14.2 

14.6 

-0.5 

0.741 

College courses 

6.8 

4.0 

2 g*** 

0.004 

Vocational or technical 

6.4 

4.6 

1.9* 

0.054 

Other 0 

6.2 

3.1 

2 2 *** 

0.000 

Sample size (total = 2,845) 

1,852 

993 




SOURCES: MDRC calculations based on responses to the 12-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
GED = General Educational Development. 

“"Vocational school" includes technical, business, and trade schools. 

b "Trade license/training certificate" includes technical, business, and trade certificates. 

“"Other" category includes enrollment in a regular or special education high school or a charter school. 
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Appendix Table D.2 


Impacts on Youth Development at 12 Months 


Outcome 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Civic engagement since random assignment (%) 

86.4 

75.7 

10 . 6 *** 

0.000 

Volunteered 

47.8 

20.4 

27 4 *** 

0.000 

Registered to vote 3 

76.3 

71.5 

4 g*** 

0.006 

Voted 

35.2 

31.3 

3 9 ** 

0.038 

Involved in politics or local community activities 

10.9 

7.2 

3 7 *** 

0.003 

Currently happy (%) 

76.6 

75.2 

1.4 

0.418 

What the future holds (%) 





Willing to wait for bigger financial rewards ' 3 

30.8 

28.3 

2.5 

0.191 

Will probably attend college 0 

83.4 

81.7 

1.7 


Expects to live at least 70 years 

78.5 

78.0 

0.5 

0.787 

Exhibits signs of major depression 0 ’ (%) 

18.0 

17.3 

0.7 

0.644 

Overall good health (%) 

84.5 

82.8 

1.7 

0.267 

Believes most people can be trusted (%) 

23.2 

21.5 

1.7 

0.328 

Social support scale 6 

3.1 

3.1 

0.0 

0.160 

Self-esteem scale f 

3.3 

3.2 

0 . 0 * 

0.097 

Self-confidence scale 8 

3.0 

2.9 

0.0 

0.180 

Sample size (total = 2,845) 

1,852 

993 




SOURCES: MDRC calculations based on responses to the 12-month survey. 


NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

“Currently registered to vote at the time of the survey. 

b Based on responses to the question, "Would you rather get $80 tomorrow or get $100 three months from 
now?" 

“Among those who have not attended and are not currently attending college. 

^Depression is measured using the PHQ-9, a nine-item scale used to diagnose depression in clinical settings. 
Response categories range from 0 = "not at all" to 3 = "nearly every day," where higher scores indicate more 
frequent occurrence of depression symptoms. If the item score sum is greater than or equal to 10 , the respondent is 
considered to exhibit signs of major depression. 

“Social support is measured using a six-item scale. Response categories range from 1 = "strongly disagree" to 
4 = "strongly agree," where higher scores indicate stronger social support. The six items are averaged. 

‘Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 
1 = "strongly disagree" to 4 = "strongly agree," where higher scores indicate higher levels of self-esteem. The 10 
items are averaged. 

8 Self-confidence is measured using a six-item scale. Response categories range from 1 = "strongly disagree" to 
4 = "strongly agree," where higher scores indicate higher levels of self-confidence. The six items are averaged. 
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Appendix Table D.3 

Impacts on Employment and Earnings at 12 Months 


Outcome 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Ever employed since random assignment (%) 

54.5 

55.9 

-1.4 

0.497 

Currently employed (%) 

33.2 

31.7 

1.4 

0.455 

Working full time (35+ hours/week) 

15.2 

15.4 

-0.2 

0.916 

Self-employed 

7.9 

8.7 

-0.8 

0.475 

Working through a temp agency 

5.6 

5.2 

0.4 

0.657 

Earning $ 10/hour or more 

8.3 

8.0 

0.3 

0.783 

Average weekly earnings ($) 

92.3 

90.2 

2.1 

0.777 

Current job industry (%) 

Construction 

2.2 

2.8 

-0.6 

0.327 

Retail trade 

4.4 

6.0 

-1.7* 

0.069 

Admin./support/waste mgmt./remediation 

3.9 

3.7 

0.2 

0.807 

Health care and social assistance 

5.0 

3.5 

1.5* 

0.088 

Accommodation and food service 

7.9 

8.4 

-0.5 

0.647 

Other 

9.1 

6.4 

2 j** 

0.018 

Sample size (total = 2,845) 

1,852 

993 




SOURCE: MDRC calculations based on responses to the 12-month survey. 

NOTE: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
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Appendix Table D.4 


Impacts on Living Arrangements and Household Information at 12 Months 


Outcome (%) 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Current living arrangement 





Parent's home 

52.6 

56.1 

-3.5* 

0.072 

Another person's home a 

22.8 

20.6 

2.2 

0.196 

One's own place 

19.7 

17.0 

2.7* 

0.077 

Incarceration facility 

1.5 

1.5 

-0.1 

0.876 

Other arrangement b 

3.4 

4.8 

-1.3* 

0.097 

Ever homeless since random assignment 

16.4 

16.5 

-0.1 

0.946 

Married or living with spouse/partner 

22.7 

21.7 

1.0 

0.569 

Receiving government benefits 

72.0 

70.3 

1.7 

0.361 

Has children c 

40.6 

41.8 

-1.3 

0.375 

Young parent’ 1 

7.3 

7.5 

-0.2 

0.829 

Lives with all or some of one's children 

29.8 

28.9 

0.9 

0.525 

Has children but not custody 

13.2 

15.0 

-1.8 

0.167 

Paid child support in the last 30 days 

2.0 

2.1 

-0.1 

0.810 

Sample size (total = 2,845) 

1,852 

993 




SOURCE: MDRC calculations based on responses to the 12-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
“Includes living with family other than parents. 

'’Includes living in a group home or halfway house, a long-term homeless shelter, or an emergency 
housing shelter (including for domestic violence); living on the street; situations such as college or residen- 
tial training programs; and other situations. 

“Includes a person's biological, adopted, foster, and stepchildren, plus any other children he or she is 
responsible for. 

d " Young parent" is defined here as a person under the age of 20 who is pregnant or who has a child. 
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Appendix Table D.5 

Impacts on Delinquency and Risky Behavior at 12 Months 


Outcome (%) 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Arrested 

18.4 

16.9 

1.5 

0.341 

Charged 

16.2 

15.3 

0.8 

0.576 

Convicted or found delinquent 3 

9.8 

8.4 

1.4 

0.230 

Drag offense 

2.7 

2.5 

0.2 

0.775 

Driving under the influence 

0.6 

0.7 

-0.1 

0.713 

Failure to pay child support 

0.1 

0.0 

0.1 

0.241 

Property offense b 

3.3 

2.5 

0.8 

0.280 

Violent offense 0 

1.8 

1.3 

0.5 

0.357 

Other 

1.8 

1.3 

0.5 

0.386 

Ever locked up due to a sentence 

5.1 

3.9 

1.2 

0.167 

Involved in a gang fight in the past 12 months 

3.8 

5.0 

-1.2 

0.161 

Substance abuse 

bias 5+ drinks once or more in a typical week 

30.0 

29.0 

1.0 

0.606 

bias used marijuana since random assignment 

35.2 

36.8 

-1.5 

0.429 

Has used another drug since random assignment 

8.6 

6.5 

2.0* 

0.067 

Drove a car while drinking or doing drugs 

in the last 30 days 

5.7 

5.2 

0.6 

0.558 

Sample size (total = 2,845) 

1,852 

993 




SOURCES: MDRC calculations based on responses to the 12-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

All outcomes reflect activity since random assignment unless otherwise noted. 

Tn the juvenile justice system, the term "adjudicated delinquent" is used rather than "convicted." 
b Property offenses include shoplifting, burglary, larceny, theft, auto theft, writing bad checks, fraud, forgery, arson, 
vandalism, and possession of stolen goods. 

c Violent offenses include physical or sexual assault, rape, robbery, manslaughter, attempted murder, and murder. 
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Appendix Table D.6 
Impacts by Age at 12 Months 


Outcome 

Sample Members Under 20 


Sample Members 20 and Older 

Difference 

Between 

Subgroup 

Impacts 

YouthBuild Control Difference 

Group Group (Impact) 

YouthBuild Control Difference 

P-Value Group Group (Impact) 

P-Value 

Ever participated in YouthBuild (%) 

73.1 

NA 

NA 


74.1 

NA 

NA 



Graduated from YouthBuild (%) 

47.5 

NA 

NA 


52.0 

NA 

NA 



Service receipt (%) 










Education-related 

76.2 

62.1 

\ *** 

0.000 

73.9 

52.4 

2i 4*** 

0.000 

f 

Job- or training-related 

69.2 

39.4 

29 9*** 

0.000 

72.6 

39.4 

23 2*** 

0.000 


Personal development 

58.9 

33.6 

25.3*** 

0.000 

60.0 

29.0 

31.0*** 

0.000 


Education (%) 










Earned a GED since random assignment 

25.3 

12.0 

^3 2*** 

0.000 

27.1 

10.6 

16.5*** 

0.000 


Enrolled in college since random 










assignment (2- or 4-year) 

14.0 

9.6 

4.5** 

0.023 

17.7 

12.8 

5.0** 

0.020 


Youth development 










Civic engagement 3 (%) 

83.5 

70.5 

13.0*** 

0.000 

89.5 

80.6 

g 9*H=H= 

0.000 


Self-esteem scale* 3 

3.2 

3.2 

0.0 

0.389 

3.3 

3.2 

0.0 

0.136 


Work (%) 










Currently employed 

33.4 

29.7 

3.7 

0.186 

32.6 

33.9 

-1.3 

0.622 


Earnings greater than $10 per hour (%) 

7.2 

7.8 

-0.7 

0.673 

9.3 

8.4 

0.9 

0.580 



(continued) 



Appendix Table D.6 (continued) 



Sample Members Under 20 


Sample Members 20 and Older 


Outcome 

YouthBuild Control Difference 

Group Group (Impact) 

P- Value 

YouthBuild Control Difference 

Group Group (Impact) P-Value 

Difference 

Between 

Subgroup 

Impacts 

Involvement in the criminal justice system 
Arrested since random assignment (%) 

20.3 17.2 3.1 

0.180 

16.1 16.7 -0.6 0.781 


Sample size 

980 475 


872 518 



SOURCE: MDRC calculations based on responses to the 12-month survey. 


NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The H-statistic is used to assess whether the difference between the supgroup impacts is statistically significant. Statistically significant differences 
between subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

NA = not applicable. 

a "Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, or being involved in politics or local 
community services. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 1 = "strongly disagree" to 4 = "strongly 
agree," where higher scores indicate higher levels of self-esteem. Responses to the 10 items are averaged. 



Appendix Table D.7 


Impacts by Gender at 12 Months 




Women 



Men 



Outcome 

YouthBuild 

Group 

Control Difference 
Group (Impact) 

P-Value 

YouthBuild Control Difference 
Group Group (Impact) 

P- Value 

Difference 

Between 

Subgroup 

Impacts 

Ever participated in YouthBuild (%) 

72.9 

NA 

NA 


74.1 

NA 

NA 



Graduated from YouthBuild (%) 

44.3 

NA 

NA 


52.8 

NA 

NA 



Service receipt (%) 










Education-related 

75.4 

60.4 

15.0*** 

0.000 

74.6 

55.7 

Jg 9*H=H= 

0.000 


Job- or training-related 

68.0 

36.7 

31.3*** 

0.000 

72.9 

40.3 

32.6*** 

0.000 


Personal development 

61.8 

31.1 

30 7 *** 

0.000 

58.3 

30.9 

27 zj.*** 

0.000 


Education (%) 










Earned a GED since random assignment 
Enrolled in college since random 

21.8 

10.6 

H 1*** 

0.000 

28.8 

11.8 

15 9 *** 

0.000 

t 

assignment (2 or 4-year) 

16.8 

12.0 

4.9* 

0.052 

15.1 

11.1 

4.0** 

0.024 


Youth development 










Civic engagement 3 (%) 

88.5 

75.6 

12 g*** 

0.000 

84.9 

76.0 

g 9 *** 

0.000 


Self-esteem scale b 

3.2 

3.3 

0.0 

0.860 

3.3 

3.2 

0 . 1 * 

0.055 


Work (%) 










Currently employed 

30.6 

28.7 

1.9 

0.560 

34.8 

33.6 

1.3 

0.611 


Earnings greater than $ 1 0 per hour (%) 

5.9 

5.2 

0.7 

0.668 

9.7 

9.7 

0.0 

0.984 
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Appendix Table D.7 (continued) 




Women 



Men 



Outcome 

YouthBuild 

Group 

Control Difference 
Group (Impact) 

P- Value 

YouthBuild Control Difference 
Group Group (Impact) 

P- Value 

Difference 

Between 

Subgroup 

Impacts 

Involvement in the criminal justice system 
Arrested since random assignment (%) 

11.8 

9.2 2.6 

0.241 

22.1 

22.2 0.0 

0.988 


Sample size 

697 

366 


1,152 

627 




SOURCE: MDRC calculations based on responses to the 12-month survey. 


NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The H-statistic is used to assess whether the difference between the subgroup impacts is statistically significant. Statistically significant differences 
between subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

NA = not applicable. 

a "Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, or being involved in politics or local 
community services. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 1 = "strongly disagree" to 4 = "strongly 
agree," where higher scores indicate higher levels of self-esteem. Responses to the 10 items are averaged. 



Appendix Table D.8 

Impacts by Highest Grade Previously Completed, at 12 Months 


Sample Members Who Had Completed Sample Members Who Had Completed 

Less Than Tenth Grade Tenth Grade or Higher 


Outcome 

YouthBuild 

Group 

Control Difference 

Group (Impact) 

P- Value 

YouthBuild 

Group 

Control Difference 

Group (Impact) 

P-Value 

Difference 

Between 

Subgroup 

Impacts 

Ever participated in YouthBuild (%) 

74.7 

NA 

NA 


72.6 

NA 

NA 



Graduated from YouthBuild (%) 

51.0 

NA 

NA 


47.6 

NA 

NA 



Service receipt (%) 










Education-related 

77.5 

60.0 

17 5*H=H= 

0.000 

72.7 

54.9 

yj g*** 

0.000 


Job- or training-related 

72.1 

37.3 

34.8*** 

0.000 

69.6 

41.6 

27 9 *** 

0.000 

f 

Personal development 

60.9 

32.0 

2 g 9 *H=H= 

0.000 

57.9 

30.1 

27 7 *** 

0.000 


Education (%) 










Earned a GED since random assignment 

27.5 

10.4 

17 1*H=* 

0.000 

24.2 

10.2 

140*** 

0.000 


Enrolled in college since random 










assignment (2- or 4-year) 

14.0 

8.1 


0.001 

17.8 

13.8 

3.9* 

0.084 


Youth development 










Civic engagement 3 (%) 

84.0 

71.5 

12 5*H=* 

0.000 

88.9 

80.8 

g QH=*H= 

0.000 


Self-esteem scale b 

3.2 

3.2 

0.0 

0.938 

3.3 

3.2 

0 . 1 ** 

0.013 

f 

Work 










Currently employed (%) 

31.6 

28.4 

3.2 

0.223 

34.7 

36.0 

-1.3 

0.668 


Earnings greater than $ 10 per hour (%) 

7.0 

7.9 

- 0.8 

0.587 

9.7 

8.4 

1.3 

0.475 



(continued) 



Appendix Table D.8 (continued) 



Sample Members Who Had Completed 

Less Than Tenth Grade 

Sample Members Who Had Completed 

T enth Grade or Higher 


Outcome 

YouthBuild 

Group 

Control Difference 

Group (Impact) 

P-Value 

YouthBuild Control Difference 

Group Group (Impact) 

P- Value 

Difference 

Between 

Subgroup 

Impacts 

Involvement in the criminal justice system 
Arrested since random assignment (%) 

18.7 

18.5 0.1 

0.960 

17.6 15.8 1.8 

0.437 


Sample size 

1,005 

525 


821 456 




SOURCE: MDRC calculations based on responses to the 12-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The H-statistic is used to assess whether the difference between the subgroup impacts is statistically significant. Statistically significant differences between 
subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

NA = not applicable. 

a "Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, or being involved in politics or local community 
services. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 1 = "strongly disagree" to 4 = "strongly agree," where 
higher scores indicate higher levels of self-esteem. Responses to the 10 items are averaged. 



Appendix Table D.9 

Impacts by Program Fidelity, at 12 Months 


Outcome 

Sample Members from Programs with 

Low Fidelity 

Sample Members from Programs with 

High Fidelity 

Difference 

Between 

Subgroup 

Impacts 

YouthBuild 

Group 

Control Difference 

Group (Impact) 

P-Value 

YouthBuild 

Group 

Control Difference 

Group (Impact) P-Value 

Ever participated in YouthBuild (%) 

68.7 

NA 

NA 


76.7 

NA 

NA 



Graduated from YouthBuild (%) 

55.8 

NA 

NA 


46.0 

NA 

NA 



Service receipt (%) 










Education-related 

75.6 

59.7 

16.0*** 

0.000 

74.8 

55.7 

jp JH=H=H= 

0.000 


Job- or training-related 

69.1 

44.0 

25 J*H=H= 

0.000 

72.0 

35.7 

36.3*** 

0.000 

ftt 

Personal development 

57.1 

33.0 

24.0*** 

0.000 

61.1 

29.7 

31.3*** 

0.000 

t 

Education (%) 










Earned a GED since random assignment 

28.5 

12.3 

16.2*** 

0.000 

24.2 

11.2 

13.0*** 

0.000 


Enrolled in college since random 










assignment (2- or 4-year) 

19.5 

13.1 

6.5*** 

0.007 

13.6 

9.7 


0.028 


Youth development 










Civic engagement 3 (%) 

87.2 

76.5 

7*H=H= 

0.000 

86.0 

75.4 

10.6*** 

0.000 


Self-esteem scale b 

3.3 

3.2 

0.1* 

0.082 

3.3 

3.2 

0.0 

0.488 


Work 










Currently employed (%) 

31.6 

32.5 

-0.9 

0.757 

33.9 

31.1 

2.8 

0.271 


Earnings greater than $10 per hour (%) 

10.4 

8.0 

2.4 

0.207 

7.2 

7.3 

-0.1 

0.971 



(continued) 



Appendix Table D.9 (continued) 



Sample Members from Programs with 

Low Fidelity 

Sample Members from Programs with 

High Fidelity 


Outcome 

Program Control Difference 
Group Group (Impact) 

P- Value 

Program Control Difference 
Group Group (Impact) 

P- Value 

Difference 

Between 

Subgroup 

Impacts 

Involvement in the criminal justice system 
Arrested since random assignment (%) 

18.0 14.7 3.2 

0.177 

18.5 18.5 0.0 

0.989 


Sample size 

723 428 


1,129 565 




SOURCE: MDRC calculations based on responses to the 12-month survey. 


NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The H-statistic is used to assess whether the difference between the subgroup impacts is statistically significant. Statistically significant differences 
between subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

NA = not applicable. 

a "Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, or being involved in politics or local 
community services. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 1 = "strongly disagree" to 4 = "strongly 
agree," where higher scores indicate higher levels of self-esteem. Responses to the 10 items are averaged. 



Appendix Table D.10 

Impacts Per Participant for Selected Outcomes 


Outcome 

YouthBuild 

Group 

Control 

Group 

Difference 

(Impact) 

Impact Per 
Participant 

Education and training (%) 

Earned a GED 

31.2 

17.5 

13.7 

18.5*** 

Enrolled in vocational school 

30.8 

20.4 

10.3 

14 0 *** 

Received a trade license/training certificate 

4.1 

2.1 

2.0 

2 7 ** 

Enrolled in postsecondary courses 

23.6 

18.1 

5.6 

7 5 *** 

Work and earnings 

Employed at Month 30 (%) 

44.4 

44.8 

-0.4 

-0.5 

Average weekly earnings ($) 

150.2 

134.5 

15.7 

21.2* 

Youth development 

Civic engagement 3 (%) 

92.2 

88.6 

3.6 

4 g *** 

Self-esteem scale b 

3.3 

3.3 

0.0 

0.0 

Criminal justice involvement 

Arrested since random assignment 

27.6 

26.4 

1.3 

1.7 

Convicted since random assignment 

15.7 

14.1 

1.6 

2.2 

Sample size (total = 2,808) 

1,830 

978 




SOURCE: MDRC calculations based on responses to the 30-month survey. 

NOTES: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

"Impact per participant" refers to the difference between the program and control group means divided by the 
participation rate (0.74). 

a "Civic engagement" covers volunteering, being registered to vote at the time of the survey, having voted, and 
being involved in politics or local community services. 

b Self-esteem is measured using the 10-item Rosenberg Self-Esteem scale. Response categories range from 
1 = "strongly disagree" to 4 = "strongly agree," where higher scores indicate higher levels of self-esteem. 
Responses to the 10 items are averaged. 
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